As others note-- he doesn't really have a leg to stand on with his either mean or just plain ignorant claim he should get compensatory time.
But there is a more troubling aspect to what you posted.
You post that your ex refuses to "uphold any pre-existing plans that Chickpea had." I take that to mean that if she was invited to another kid's birthday party and it's his weekend, for instance, he will not take her himself because it interferes with "his time" even if it's something SHE really wants and expects to do. Or if she's enrolled in a weekly class of some sort--dance or art or music--or if she's (eventually) doing a sport, he will just say, "I will not take you." Is that what he means?
I don't know how old your daughter is now, but as she gets older she will have more and more activities and SHE will want to do them regardless of "his time" or "your time." A child can't do soccer practice and games only every other weekend, or dance class only two Saturdays a month instead of all four; she will never advance in any of her interests and skills, and she will end up being told, "Sorry, you can't play in next week's game/perform in the recital/etc. because you weren't at practice/rehearsal enough times." It does happen and it will happen to her unless daddy grows up before she gets involved in activities. Imagine how that would affect her; she might just decide not to pursue her interests because it's not worth being the kid who is only there some of the time, and who is always a little behind the other kids because of that, and who doesn't get to perform or play like the other kids.
And he is pretty dumb if he doesn't realize what a jerk and villain that is going to make him in her eyes as she gets older. Picture her telling her classmate in a few years: "Oh, no, your birthday party's weekend AFTER next?! Oh...I'm with my dad then. No, he won't let me come." That is going to get very old very fast, and is a recipe for a kid to start resenting and rebelling sooner rather that later. He could be the good guy, good dad so easily but he's deciding he prefers slapping at you to supporting her.
It hasn't happened in my family but I sure did see it with a friend whose ex-husband did the same routine of "my time is sacred" and refused to take their son to anything --until the son got into a particular sport that dad liked, and then it was just dandy with dad.
I'm sorry to read that you agreed with him that his time is so sacred (I'm sure you did so that you can say the same about your own time with her, which makes sense). But when she is older and wants a say in her own time -- whichever parent she's with--what happens then? If you and he can't agree that "She's signed up for art class that meets every Saturday and on my weekends, I take her, on your weekends, you take her" -- then she will be the kid who never gets to follow through on her interests or make friends through her activities. .
You and he might need to see a mediator eventually to discuss your daughter's participation in regularly scheduled extracurricular activities and parties and things that she wants to do. I"m not talking about the family reunion or something that's strictly your family event or his family event, but about things with her friends or activities she wants to do. '
It sounds as if your ex is using this to slap at you and show that he controls your daughter when she's with him. He doesn't know (or care?) that eventually it will hurt your daughter much more than it hurts you. I am guessing that he sees any "pre-existing plans" as YOUR plans and likes the idea of getting under your skin and defeating you by refusing to follow your plans. He needs to find out that eventually this will hurt your child and make it hard for her to find her own interests and to stick with things. Please consider bringing this up with a mediator if he's too immature and stubborn to have an adult conversation with you about it.