Hmmm. 4 hour drive. Yes, they should make it to important things (funerals, weddings), and they should try to see you guys once or twice per year for sure. But I'm in the camp that every little child's birthday is not important to adult relatives. I never invite family to my kid's birthdays (they live far away) just friends. Also, I didn't have one year birthday parties and two year birthday parties-or even 3 year ones aside from little celebrations at home. So if someone invited me to a one year old's birthday party 4 hours away, I wouldn't even entertain possibly going. Not that every life isn't precious, but you get my drift.
What if they came to ALL your kids' birthdays? Wouldn't that be a bit excessive at 4 hours away if you guys were also going to see them at their stuff?
I would only count the big stuff, and even then, I would base it on how many times they see you on average per year, not which events.....
Now if you make sure to do all their kid's birthdays and every little event, and they get mad if you ever miss anything, then that's not fair. So you do need to even the playing field. But I can tell you, I have NO friends or family who expect me to be at all those events, and I could never manage it or expect it of them either.
If they NEVER get together with you, then yes, you need to stop making an effort for them as well.
My husband has a twin brother in another state. They've always been close. They still are. Sometimes years go by when they haven't seen each other. Sometimes they don't speak other than emails for a year or so. The last time my husband visited that part of his family his brother went out of town with his wife so they didn't even see each other. Was that a problem for either of them? no. They're brothers.
Sounds like you guys are super sensitive about this and need to have open communication about it with their family so you guys can work it all out.