I never wear shoes unless I have to. I try to get away with not even wearing socks in the winter. I take my bra off the second I get into the house, and my husband takes his shirt off, so that he's only wearing his undershirt.
As far as your pants coming off...My dad did that when I was little, and I totally HATED it!! Like I really wanted to grow up seeing my dad in his underwear all the time...yuck! Around age 5 I started to ask him to wear pants, but he never did. I think I remember your children being toddlers, so I don't see it as A. issue for them. When they get older, I'm willing to bet they really won't want to see their mom pant-less all the time. Why not keep a pair of pants close to the door, so you can slip them on quickly? You might think of teaching them they can only drop their pants at home, and not anywhere else. That's definitely not the norm, and they probably don't want to be doing that at other houses :)