I am having the same problem there is lack of respect for space when it comes to babies. When I go to the grocery store and reach for something in an aisle almost every visit to the store when I turn around someone is touching my babies hands or asking to hold them.
Family and friends are not a problem because my baby was in a NICU I ask when they reach for the baby or ask to hold him if they are sick and if they have washed their hands because the baby cannot get sick and I have Purell with the pump in each room so that if they say there hands are clean but I have not seen them wash I pump Purell into their hands as they reach for the baby.
In regards to strangers I tell them oh no don't touch my baby I think he is coming down with something when other kids are trying to touch him when adults ask to hold and touch him I tell them that he has some anxiety about strangers and then I pat his back and say it's ok baby and walk away.
I do not feel bad about doing these things because if I were actually being upfront and as rude as them I would say I don't know you, you are a stranger do not touch my baby or let your children sick or well touch my infant.
The best thing to do is start telling the kids and parents no now otherwise each time they see the baby they will want to kiss and touch and hold the baby you are not over reacting my friends with kids totally understand and do not push themselves on me if my friend were sick and asked to hold my baby I would tell her "you are sick and the baby can't get sick so you can hold him later or you will have to come over and take care of him when he is sick". I have told my parents and in-laws the same if you are sick don't come over and/or when you are feeling better you can hold the baby as you do not want him sick.