My nephew is very big for his age and always has been. He'll be 9 this summer, and he's close to 5 ft and over 130 lbs. Everyone in their family is big - tall and very solid. It's hard for us, because our son was born 1 year and 1 day after his cousin and is small for his age. So the two cousins easily appear to be 3 years apart if not more.
At 36 inches he is a bit off the chart but just a bit - the chart I looked at said 34 inches. It's hard for me to believe an extra 2 inches is really that big of a deal, though I didn't go to med school. I suspect your doctor is just being cautious.
Other than shopping for clothes in the boys section sooner than other moms and possibly having some of the social challenges Fanged Bunny mentioned, I really think this is not a big deal. Your son is just going to be one of the tallest in his class.
It sounded like you might be concerned that he isn't talking much? Maybe not, but if you were thinking that at all I would just say that 17 months is very young. It's not at all unusual for boys, especially a second born, to not be saying much at 17 months. That being said, my son wasn't saying much either and I insisted on a speech evaluation. He's been going to speech since he was about 19 months old and is now 5 and doing fabulous. Not really something to be too concerned about, but something to keep an eye on.