By the way you describe it, it sounds to me that it's exactly what you said... he wants to get his, with little regard for what you are getting out of the deal. And that doesn't cut into his game time. I'd be pi$$ed.
Obviously, you can't take much time to do the deed with a toddler and a preschooler running around, unsupervised, not to mention that you greatly increase the risk of them interrupting you when they're up, and you've gone missing...
I'd at least try to work with him on a compromise... yes, it may seem more 'scheduled' but so what if that's what it has to be for you to get your intimate time, and some time to "get your rocks off" too, which you absolutely should! I'd continue to appease him on the daytime quickies, since that's really all they can be with 2 awake little ones, but I'd ask him to compromise on the night time intimacy too. Maybe talk to him about how you need more time with him, that you want to be close to him for more than a wham, bam, thank you mam... and hopefully he'll oblige you, as well... I guess I'm saying it doesn't have to be all or nothing either way... a compromise somewhere in the middle should keep everyone relatively happy, no?