J.D.
It's ok... don't beat yourself up over it. I'll give you my situation from the other day. For some reason 3 of the women on my block don't watch their kids (ages 2 - 6) that closely. It drives me nuts... These kids just run everywhere.
One of the 3 year old boys has a mean streak and keeps targeting my daughter (17 mos). He walks over to her and pushes her down for no reason. Elizabeth doesn't cry, but she is confused and I don't want her to learn this behavior. So after the 3rd time (all different days), I had it... I said "NO" in a loud voice. The 3 year old started crying and went to his mother. I apologized and she made the statement that some women are just overly protective of their first babies. I told her "No, that is the 3rd time and he needs to stop." I picked up Elizabeth and went inside. I haven't been out to play with that kid again.
I'm sure I will be back out there sometime, but I'm not ready yet. So, I guess what I'm saying is you did what your gut told you to do. Let it go. It will blow over. We can't raise other peoples' kids no matter how frustrating it is, but we can protect our own.
GL!