Unless you see her crying and upset I suggest you not take the comment seriously. If your daughter is happy when you pick her up, I would ignore the ex's comments. She can be telling you this to upset you and various other reasons.
My two youngest granddaughters, at that age, cried when I picked them up and when they got back home. I saw this and knew she was having a difficult transitioning. At 3 they really don't know what is happening. My granddaughters trusted me and her parents. They were excited to come with me until parent's put on jackets and shoes. Then they sometimes had a meltdown before we left. They were OK once we got in the elevator.
I suggest that the only time to pay attention to your ex's explanation is when YOU see daughter having difficulty.
If you think the ex is trying to build a case for changing custody, get ahead of the game by talking with your lawyer, asking if you need to do something.
Do you and your ex not have regular, free of anger and blame conversations?
I would ask him how he sees the visiting going.
I would ask that her Dad be present some of the time. Because ex is not married, the girlfriend is not a step-mother. Most of your conversations should be with your ex. And her Dad should be the one interacting with his daughter most of the time.