My son just turned 4 at the end of August...and the way things worked out - I didn't have a 1st B-day party for him!!! There were a few family members in town to visit & spend time, but we didn't have a cake or anything - he had no idea!!! However, for age 2 - we did a big party complete with cake, friends, decorations, presents, etc. Then when his 3rd came around, my budget was just not there, so we just did a home-made cake on the front porch/steps with about 4-5 of his friends...we sang Happy Birthday, let him blow out candles, blow up balloons & open some of the few presents he got. He LOVED it! Now the 4th one that just passed, we did another party, but our budget was still on the low...so I did most of the shopping for stuff @ the Dollar Store and we went to the local park where we had food that we brought along (a few salads, home-make cupcakes & cake -again!- and beverages to share). There was free roller-skating w/music going on at the park and there's a new playground...the children all brought "wheels" (either scooters, rollerskates, bikes, etc) and they had a BLAST! There were actually about 12 children & their parents - but we had just enough of everything. It didn't cost much & everyone had a good time.
Oh...and the children that attended (ranging in ages 2-10...yes, my son has older friends!) all enjoyed the "loot" bags that I had prepared. I did a few for girls too. And again, I got all the stuff from the Dollar Store!
I guess what I'm getting at it is don't feel bad if you skip a year in having a big party...as long as you do some sort of recognition, they'll be ok. And if you want you can do every other year (?)...at this point, that seems like the route I may go...we'll see. It basically depends on the budget & what else is going on.
Hope this helped! Happy B-day (soon) to your little man!