J.
I think that's crazy, too. Do you even know the other people who are going?
You don't have a lot of time to decide, but I think you need to make a choice.
One, confront and give an ultimatum, but it seems like your husband won't back you up. So that would require you to have a talk with him.
Two, send your husband and kids while you stay home.
Three, make the best of it and put it out of your mind. Be a "guest" and don't worry about meal planning - let the other people deal with that stuff (just make sure you bring some supplies for your own kids so no one goes hungry) Make them babysit your kids one night for free while you go out with your husband. Then have as much fun as you can and decide to pretend the bad stuff didn't happen - and never plan anything with these people again! Hopefully most of the people won't stay all week and won't come all at the same time - you may be surprised and it could be a great time for all the kids and for you too.
Good luck. It really sucks to be put in this position. Don't let it ruin your week off completely one way or another, though.