I really wouldn't worry about it.
Have you ever heard the saying, "actions speak louder than words?" Well it's true.
Our kids are developing their language right now. They are discovering how to articulate their thoughts, WHILE they are discovering what their thoughts are, why they are, and what they mean. They haven't understood a word's context or value, or it's full definition.
Their feelings are raw, and overwhelming. They don't yet fully understood how to react to their emotions, nor what their emotions mean.
Take for example how a younger child will refer to all animals as either a dog, or a snake. The cat is a dog, the fish is a snake, the goat is a dog and a lizard is a snake. Everything is either a truck or a mommy. It's not because those things are true, it's because their associations of words are still developing. Likewise, a sad confused child will often behave angrily, because they don't know how to express their feeling, they don't know how to be heard, their frustrated and overwhelmed about the feeling itself, etc. So a doll gets taken away from one child by another child, and the child hits/screams at/pushes/etc. the offender.
They are also gaging reactions.
They are learning to be grownups. We are teaching them how to be adults. But they are still kids.
My children (almost three, just turned four) have said this. For example, my niece (when she first came to live with us) would say, "I don't love you, I love my mommy!" What she meant, was, "I'm angry that my mommy is gone. I blame you, because and I'm confused and scared. I miss my mommy. Why is my mommy gone?" But she doesn't yet know that those are her feelings, nor how to express them. Because she was three years old, not 30 years old.
I responded, "You can love your Mommy and me. I love your mommy and I miss her too. I feel sad when you say you don't love me, but I love you and I will no matter what. I hear you feel angry. You get to be angry. You don't get to be mean to (my daughter's name) or me because you are angry. You can ask for a hug and you can talk to me, if you'd like."
It took about a month, and then she stopped saying it. (we implemented many more techniques, and time ins (not outs), and based discipline around positive reinforcement alone. because the root of her anger and behavioral issues was deep trauma and grief. It was necessary to become more permissive while she regressed/healed - sort of a building back up again).
My daughter just said this a few days ago. She was upset because I told them we were done reading bedtime stories. She said, "me no love you mommy." What she meant was, "I am upset with you because I want to read more, and you are telling me no. I feel powerless." Here, I said, " (insert name), saying that is hurtful. I know that you love me, and saying that you don't, because you are frustrated, is not okay. You are allowed to be frustrated, but you are not allowed to treat me poorly." She said, "Okay mommy. I sorry."
And that was that.
In all situations, it's helpful to name their feelings for them, because they learn how to recognize their emotions and how to express them. "I see you feel sad when the dog popped your balloon," or, "I hear you feel angry when ____ ran into you," etc.
He's just learning language and what it means. He's not ACTUALLY meaning that he doesn't love his daddy. I know it's rough to hear, but try not to take it too seriously.