J.K.
It sounds like you and your husband need to get on the same page.
First, if he gets a text invite, he needs to tell you that same day so you can discuss it. My husband and I have a linked Google Calendar. We can each see what the other adds to it. Super handy.
Second, he needs to address the last-minute invites with his family members by saying that if they really want you all to be able to attend, they need to ask in advance, not the day-of. If they get offended, that is their choice and it should not be allowed to impact your household.