How old is your son? When they're younger (K-2), you'll find that a home-based life works just fine. When they get a littler further along, you'll find that no one is around to play with because everyone is at sports or scouts or whatever. So if you want you kid to have a social life, you're going to have to get out of your comfort zone. It's his childhood, and he only gets one, so be sure that you're letting him lead the way, within reason, regarding how much time he wants to spend pursuing interests.
Your town may have a recreation department that runs low-commitment weekly activities that run for 6-8 weeks. Many YMCA's off this kind of thing as well, and they can be a great way to try things without having to run around every day, buy a ton of equipment, etc. When my kids were younger and dabbling in different sports, they tried basketball, karate, t-ball, lacrosse, flag football and dance through the rec department or the Y.
How busy is my family? Too busy, but we have 4 kids and that's the way it goes with that many. My husband and I strongly feel that their opportunities shouldn't be limited just because they happen to have a lot of siblings. Over the years, they've tried a wide variety of sports and activities.
Right now, my oldest child (and only girl) is 17 and does mixed martial arts as her sport and is heavily involved in extra-curriculars at school like the Green Team and Amnesty International. She was on the robotics team last year, which was a huge commitment but well worth it because they competed in St. Louis in a national championship, which was the experience of a lifetime. She also works at a restaurant and likes to save her money to travel. She's been to California, Mexico and Montreal.
All 3 of my sons (ages 8 - 16) play hockey, which means that we are at a rink 8-10 times a week for 8 months a year. Sometimes we'll have 7 games or practices from Friday - Sunday. My oldest has a job and plays guitar, my 10 year old plays percussion in the school band and takes piano lessons, and both of my younger boys play lacrosse in the spring and go to various camps during the summer. My 10 year old wants to also play football in the fall but I just don't think that's going to happen with his hockey schedule.
We know families with 1 child who are as busy as we are with 4, which I think is crazy, but if it works for them, then I guess it works for them. Some of my younger son's friends were playing hockey, fall soccer and basketball at the same time for a few weeks in November. There were two boys he knows who would literally play three sports on Saturday mornings, going to soccer at 7 AM, hockey at 9 and basketball at 11, and they weren't even near each other!
At the end of the day, you really have to just let your kids do what they love, within the confines of your budget and schedule. If your son gets really interested in and passionate about something that you have the time and money for, are you really going to say no, that's eating into our game night or picnic time? Sports, music and the arts can be "family time" too - you spend a lot of time in the car chatting, there are meals before or after activities, you spend time throwing a ball or helping your child run scales on the piano, etc. It's not all or nothing.