L.R.
K., could you please post some details and examples so we know what you mean? I don't quite understand how these parents are not letting their kids "hang out" with yours; do you mean that you can't get kids to come over for play dates, or your kids never get invited to play dates, parties, etc.? Or are you referring mostly to other kids not befriending your kids at school? You wrote that "they won't let their kids be involved only at school with my kids" and that's not clear to me. Do you mean they will only let their kids be around yours AT school but not when you invite their kids over? It would help to know what is going on in more detail.
Meanwhile, is it possible that families are just so swamped with school; sports; scouts; dance; church; and/or a million other activities that it is really not as common as you think for any kids to do stuff together outside school much, unless they're near neighbors? Around here, school buddies sometimes rarely get together outside school because everyone is on the go all weekdays after school and on weekends too, with activities and family things. It may not be the fact that parents are slighting your family personally at all -- the issue may be that the kids you are interested in having your kids pal up with are simply scheduled like crazy. It may have nothing at all to do with your having a larger family (unless you've had direct comments that that is the case).
But it's hard to say without knowing a little more. Thanks.