Feminist in all but name

It never fails to take me aback when people say "I'm not a feminist but.." and then follow the "but" with an opinion that indicates that the person is quite likely to agree with the basic principles of feminism, in the sense of believing in political, economic, and social rights for women.
 
OK, so there are worse things than this; it is not difficult to think of many worse things than people who are feminists-in-all-but-name.

Even so, I was surprised the other day at an Administrative Meeting, when a female administrator whom I had not previously met came up to me and said:

 "The Vice-Person for Stuff told me that you are the first woman [position that I now hold in my university]. I'm not a feminist, but I think that's great. It's about time. We need more women [in that job] in the university."

She went on to say that the lack of women leaders in her unit of the university was a serious, longstanding problem. Most of the students and researchers are women, but few of the leaders are women. Every once in a while, a woman is allowed to be an interim head of something, but only until a man can be found to take over the job. She was frustrated by this, and more than a little cynical that it would change any time soon. She herself was an interim director and was certain that she would not be given the permanent position. She was sure they would replace her with a man, although she is highly qualified to keep doing this job.

Well, good thing she isn't a feminist because..

.. because why? I am having trouble finishing that sentence with anything that makes sense.

I don't even know why she chose to preface her sentence with "I'm not a feminist, but..". Why add that? If she had just said "I heard you're the first woman etc. and I think that's great", without the qualifier, what did she fear I would think? That she was a raving man-hating angry woman? That she is not automatically in favor of a woman being appointed to a position with some authority?

I did not ask her. I would very much like to help stamp out these twisted negative views of feminism/feminists, but after that strange little comment, our brief conversation focused on practical things that needed discussing in the short amount of time available.