Take a deep breath and relax :)
My girls have given me way more aggravation and trouble than my son. The drama, the gossip, the emotions, the mean girl stuff, they are 12 and (almost) 16 now and I swear having daughters is NOTHING like I expected. Oh, and one of them was a competitive gymnast for two years, talk about potential injuries!!!
My son did cub scouts, a little baseball and tennis, and as a teen he ran cross country and track. He tried wrestling (at his dad's request) but he hated it (so did I.) He is now a college student whose main interests are hiking, photography, film making and running.
What I'm saying is don't waste time worrying about it now because you just never know where your son's interests will lie. My husband's family is huge into team sports (lots of testosterone there!) and most of my nephews play pretty hard: football, wrestling, skiing, hunting, etc. My son just never had an interest in any of it, he's just a more cerebral guy. Yours may be as well, you never know.
And if he DOES turn out to be a football player? I'm sure your skin will be thicker by then and you'll be on the sidelines cheering him on :)