My son had a mild case of SPD at 2. He has now just about outgrown it. AND, at just-turned-five, he is reading at a 2nd- or 3rd-grade level. He can add and subtract with great confidence in the single digits, and he has a basic concept of multiplication. He can tell you more than you ever wanted to know about volcanoes, the solar system, the human digestive system, all sorts of things. He does have some issues with small motor coordination and overall muscle tone, and he's on the shy side (but plays well with other kids once he decides he likes them). He's still got a few lingering sensory issues, but he is in no way behind.
Now, this is just one kid. Your son will have a completely different trajectory and life story -- we all do. But if anything, SPD is associated with giftedness. I totally understand and empathize with your concerns, but I also think it's important to try to avoid comparing children with each other, or thinking that we all travel to the same point along the same pathway. The other side of the SPD coin is that your son probably has amazing powers of observation. He's just too young to know when to turn them off, so to speak.
P.S. My son was slow to hit many milestones, to the point where the What to Expect books made me cry. Then, I swear, when I wasn't looking, he taught himself how to read and has never looked back.