I basically agree with the other posters. My DD is 3 1/2, is potty trained (mostly), and has generally given up afternoon naps. I would never consider dropping her off at a b-day party alone, and I doubt her younger friends are even potty trained! I agree that you don't want to be solo in charge of all those little kiddos (esp charged up on punch and cake); besides, who would be left to take the photos? :-)
I agree that the easiest thing would be "Mommy & Me", trim the guest list, or rent a church room/public area. In St. Paul you have lots of other options too, like the Train Museum, ChuckECheese, etc, etc for party rentals. I wonder if the nature centers there would do any b-day things. Maybe call around to your favorite haunts to see what they can do for you. How about the Eagle's Nest and the parents pay their own admission?
Personally, I would respect naptimes depending upon how many kids you are inviting and how young they are, meaning that I'd do like an 11am party. Granted, you'd be doing lunch. But if the majority of your guest list naps and your child doesn't, hm, I think the polite thing to do would be to consider the majority of your guests. The meal could be as simple as hot dogs, applesauce and chips, right? It's just kids :-)
I've also read that the invitees to the party should number your child's age plus one - so for you, 4 guests. Easier said than done, I know. You don't want to offend anyone. But that could be helpful to you as well.
Anyway, try not to stress too much. It'll be great no matter what you decide :-)