Moderation is the key. You don't want to "ban" sugar, because when your child is old enough to obtain food on his own that will likely backfire and he'll go nuts for the candy.
However, you can eliminate sugar from easy sources and offer it from more healthy ones.
My 2.5 year old only eats candy on special occasions, like Halloween (a few pieces, and the rest she couldn't even have because of CD.) or Christmas (a candy cane.)
Other than that, her daily intake of sugar comes from whatever is in her healthy cereal (Chex or GF Rice Krispies), what naturally occurs in fruit, or whatever small amount is in her LF yogurt.
Because my toddler has CD (Celiac disease) she doesn't get sugar from a lot of sources other kids might, like cookies, pop-tarts, brownies, little debbies, etc. She just can't have those foods, and because she eats a largely "whole foods" diet, she hasn't developed a sweet tooth.
She doesn't drink soda or juice. She's still breastfed, and if it's not that, it's water...with ALL meals, and sometimes I flavor it with MiO, which is sugar free. She doesn't drink juice...not that I wouldn't let her, but she doesn't like it, possibly because it's too sweet.
My older child is almost 8 and again, she doesn't eat much for sugar except what occurs naturally in fruit, yogurt, etc. Again, she's allowed treats on special occasions and I am not adverse to buying her a candy bar once a month at the grocery store, but that's about it.
If your son can handle it, start him on gum. You can get sugar free gum that has a sweet taste, and that really helps. My 7 year old chews at least a few pieces of sugar free gum a day. She particularly likes these new dessert flavors they have, like Root Beer Float, Mint Ice Cream, and Apple Pie. She also likes all the mints.
I agree with Bug below that our bodies do need sugar, but it needs to come largely from natural sources as we've both suggested. Again, if he has a sweet tooth, try the gum!