Pregnancy would, for sure, make this a hard thing to hear from your beloved toddler. And this is such an important question you're asking – the implications stretch all the way into your son's adult life. Consider these questions:
Do you want your son to be able to identify and steer his own feelings as he grows up? (Children who can't do that are subject to a variety of emotional ills, from depression, to being easily "guilted" or otherwise manipulated by peers, or treacherous adults, or future romantic partners, or self-serving or unscrupulous bosses.)
Do you want your son to believe he has the power to hurt you emotionally? Or inversely, do you want him to grow up believing he's responsible for keeping other people happy? (If his babyish pronouncements strike you to your heart, imagine what he can do to you as a sometimes-angry teen. And consider the stereotypical plight of the child raised by guilt-inducing mothers.)
Do you want your little boy to grow up understanding honesty? (Being truthful about his thoughts and feelings is a huge part of honesty. He will be able to be more diplomatic about his feelings as he gains maturity, experience and empathy. He'll learn those from the consistent love and respect you and your husband show each other and your son, because children learn by example. If he's required during early childhood to hide his feelings, his experience of life will be less authentic. My own extension on this is that children shouldn't even be required to apologize until they feel genuinely sorry, otherwise, it's just an insincere social lie that allows them to go back to their preferred activities.)
Do you really believe he doesn't like/love you? Does he? (I expect in the moment they say such a thing, it feels true: kids are impulsively expressing their unhappiness with some requirement of yours, but that's because they are spontaneous and impulsive. Three minutes from now, the opposite is just as likely to be true. If we react negatively, we empower them to keep using such comments on those occasions when they think the reaction will meet some need of theirs.)
I'd ignore it, or smile and tell him it's okay, because you will always love him. My daughter, and much more recently my grandson, experimented with comments like this. Because nobody reacted, they gave up the experiment after only a few tries.