I don't know if it's a problem, but as you said he's not straining. But I'd still try to add more fruits and veggies and whole grains too, esp during the winter months.
I found that my kids were going longer without pooping and complaining every so often that it was "hard". I cut back on some of the snacks (crackers, fruit roll ups, string cheese, etc.) and made them drink some water or apple juice, have fresh fruits and veggies and now all is well.
You don't mention what your son's diet is like really. Try adding some applesauce, frozen blueberries (my kids pop those like candy), fresh fruit with vanilla yogurt and granola and I bet he'll be pooping daily.
I don't think his pooping every 3 days is abnormal, however I think more frequently is a sign of a healthier digestive system.
Just my viewpoint.
Best wishes