I've read the other responses on this thread, and I've read your question through twice. Maybe I'm not reading your post in the same way others are, but I read the remark about the weird clothes and strangeness to be about this boy who is influencing your son. Ladies, I don't think that she is saying that her son wears weird clothes and is strange.
Regardless of who is most likely Mr. Goth here, kids try to find an identity that sets them aside from others, IF they either don't want to fit in with the majority or if they CAN'T fit in with the majority. You've seen the comic "Zits", right? The boy's friend wears all the body rings all over himself and has the weird hair. You may be concerned that your son hangs with this boy because he may be gay, but it might just be that this boy has sought your son out because your son is willing to accept him for who he is, rather than reject him like other kids.
I have to say, with your son on the young side in 4th grade, and with a compliant personality, this kind of stuff is going to happen. Not every child has confidence and not every child is a leader. You are his best friend in the realm of getting him to find confidence in himself so that he won't be led down the wrong path. You mention that you put a stop to him swearing - did you sit down with the other boy and talk about it with him? Have you tried to "mother" the pair, or has your son walked away from the friendship? You can't prevent them from being friends at school behind your back, so perhaps it's better to try to manage the friendship than outright ban it.
I know that a lot of boys in certain schools are funneled into sports. Not every school is alike in this regard, and I don't know what yours is like. However, the ladies are right in that not all boys are interested in sports. It is good for you to get him into something that helps him develop his body and a regard for group activities. I agree with trying to get him into some area of martial arts. Some kids hate getting thrown to the mat, so do your homework and find one that you think your son can stomach.
As far as the gay thing is concerned, there is a reason you said it, and lots of people are jumping to conclusions about why. I don't know that you saying it has anything to do with him not being in sports, or if he is a bit effeminate, or what. We're not privy to what your son is like, and you don't tell us enough here. So I just want to say that the best thing you can do is love him and guide him in regards to taking care of his body, just as if the thought that he is gay hadn't crossed your mind. Start teaching him the facts of life, telling him how important it is to respect his body.
I would not tell the school counselor that you think maybe that he might be gay. You don't want a counselor who is biased against gays to approach him with that in mind. (Yes, there are counselors who would do that, just like there are teachers and principals who would.)
Other than what I have said, and I'm sorry this is so long, I would get your son involved in out-of-school activities as much as you can, that don't interfere with his studying. This way he will meet a lot of different people and can start widening his world in terms of meeting friends. If you can, have "get togethers" at your house, pizza night, s'mores night (or Saturday afternoons), movie night, etc. Never only invite just 2 people - 3-somes don't work well because two people end up hanging and the 3rd person (your son, maybe) is left out. You need at least 5. Having a good time with this will give your son confidence, and eventually he'll have a bigger group.
If he can sing and join the choir, those kids are great to tap for this. Also, when he gets to the point that kids are doing some acting, he can help build sets, and get to be friends with the theatre kids. Those types tend to be outgoing and want to do stuff together. And they love kids who will take the time and effort to build sets or work lights. It takes a big team to do this and the actors don't have the vehicle to act if someone isn't building sets and doing lights, so they don't look down on those kids. I doubt you have this in elementary school, but you probably will in middle school, so keep this in mind.
Good luck, with this Mama. If you help him through this, he'll have less of a chance falling in with a "bad" crowd, and perhaps you'll also be helping this other boy to not be a bad apple.
Dawn