Growing up in the midwest/east coast, big parties are very common, and fun! I remember going to birthday parties with huge buffets of food, a bar for the grown ups (oh my goodness people consumed alcohol at a party where there were kids, Mercy!), lots of kids and lots of music... It's one thing I've noticed about the PNW since moving here 10 years ago, people tend to be insular and extremely child-focused in the "the kid needs to know that we believe it's all about him/her all the time" kind of way. My parents had a large baby shower for us at a brew pub, it was a couples party and it was a blast!! And yes, there were kids there, and no they aren't all raging alcoholics now. We had a small party for our daughter's one & two year parties, then decided we wanted to have more fun and since parents would have to stay with their kids we did a big party for everyone (30 people total)for her 3rd.
You don't think the 4 year old gets that it's his special day?? Really? It sounds like you want it to be about your son too... "I really don't want to go because he won't even know we are there" If that's the case, don't go.
Do you know the family's situation? Perhaps they have lots of relatives and don't want to leave anyone out. It's easy to have a party "off site" and not have the set up/clean up that home parties have. Perhaps they ARE having a smaller party with just his closest friends but didn't want to leave out all his friends (or mom's friends-- what would you do if you found out she had this party and DIDN'T invite you??).
It sounds like you don't want to go, so find a way to let your son celebrate with her son another time.