J.R.
I'd ignore it. If they bring it up, just tell them what you said here - you wish you could invite everyone, but budgetary considerations dictate that you keep the party to only close friends and family. This situation has happened so many times with my various mom friends. As long as YOU aren't throwing it in their face that you're having to which they are not invited, you're fine. Honestly, anyone who would get that offended about not being invited really isn't worth the effort.