No, it is not. Sexism, like racism, never has a positive or neutral connotation in any context in which I have ever heard it used. A black man is not intrinsically different than a white man. Therefore, to treat him such is racist, period. A woman is, however, intrinsically different than a man. To denigrate her because she is a women is sexism. To treat her as if she were a man and refuse to acknowledge her as a woman is, arguably, also sexism. On the contrary, merely to react differently to her, but not negatively, is not sexism.
I find that separating connotations from the core meanings of words leads to clearer arguments. In your position, if someone calls you sexist and you deny that, it's not clear if you're denying treating someone differently based on their sex or if you just don't think you're mistreating the person. My stance is more likely to lead to a response of, "Yes, I'm being sexist, but not in a wrong way."
