First, you need to get the book Milk Memos. On its surface, its a book about pumping breast milk at work, but it's really about being a working mom. They talk about that whole "feeling guilty about not feeling guilty" thing, which I found so helpful.
In fact, for anyone who says something about me not being home, I tell them about this book, something along the lines of "I'm reading this funny book called Milk Memos, and it talks about how all sorts of moms feel differently about going back to work, and everyone tries to do what they need to do for their family and feel comfortable with themselves." I agree with the other commenters who pointed out that sometimes when people say something to make you feel bad about going back to work, it is because they see it as an indictment of their decision to stay home, or their wife's decision, so if you can subtly say something about how all women are different, then you seem open minded. My friend has a theory that many people find career-minded women to be judgmental. I don't think we are, but I suppose its good to remind people.
So that's my suggested response if you are one of those people who wants to be all Miss Manners and make people comfortable, which I find I am in the workplace. But the workplace is not your issue; its other people in your life. So I am here to tell you that you may find that you have the challenge (er - "opportunity") to be a pioneer.
I was so worried before I had kids because I knew that I was not SAHM material, and I was worried about how I would explain it to people, and frustrated by the fact that my husband would never have to deal with this.
Then I met the wife of one of my guy friends, right after she had kid 2. She told me about dealing with this when she went back to work, part time. She was so happy to get out of the house and go back to the profession that she had worked so hard in. Colleagues made dramatic statements like "Do you feel guilty, Do you feel terrible? Do you hate being here?" And she told me she cheerfully responded "No, actually, I'm happy to be out of the house!" I admired her chutzpah. I optimistically believe that by being perhaps the first working mom to say that to these people, she made it easier for the next working mom that comes into contact with these people.
So my Miss Manners approach is one way, but I have to admit, sometimes to change the culture, individual women have to take the Pioneer approach and be a little bit direct.