My son went through a terrible time at around 3 years old when he *hated* attending anything where other kids were present. It sucked, big time. Esp. because most of my social life during the daytime was other moms.
Don't get down on yourself, though. It is hard to be at home all day with the kids, and that feeling of difficulty can change as the kids get older, for different reasons.
Can you find one thing you'd like to do, just for yourself, at least once a week? I had a small meltdown last night when my husband came home. I was hungry, cooking dinner, and he told me his boss had taken him out to lunch at a really nice restaurant I've been hearing about for years. After that, I was ready to give my (usually terrific) guy a plate of Cold Nothing for dinner. We talked later, and I told him that I didn't really have anything for myself right now; so much of my time is family-centered. (My son is at preschool as I type right now. Thank heavens!)
So, that's my two bits-- see if you can find something you can do just for you at least once a week. If it means taking two boys out so you can get a walk, bundle them up in a stroller and go. Maybe it's going to a cafe or the library and reading, or meeting up with a girlfriend on the weekend, sans kids, for a cup of coffee or a drink. Whatever it is that fills you up-- see how you can work that into your life.
We've all been there-- you have my heartfelt empathy. I hate those moments too!