I think it is derogatory.
I mean, it is not like you are the lady standing behind her in line at a fast food joint. You work with her, she is your boss...she should know you by name, and if she does not yet she should try to get to know you by name. To me, that is the virtue of a good boss. I think that using a term like that, especially in a work setting, is a sign of a careless boss. At least say, "what is her name again?" to demonstrate that you trying to recall her name. That is the much more professional thing to do.
In the meantime, you can try to bring more attention to your work, to encourage her to get to know you more, if you have not already done so.