My older son, about to turn 5, has always had this tendency as well. One thing that has been key for us and that I didn't see you mention is the timing of bedtime.
My son has always, from the day of his birth, been prone to getting overtired. When he is overtired, he has infinitely more trouble falling asleep. Sometimes it seems he is actively fighting sleep which is super frustrating, other times he tries to fall asleep and can't. In fact, just this very evening he had tried for like half hr to fall asleep and finally over the monitor he called me and he said, "I know I need to sleep but I can't get to sleep!"
Why is this happening tonight? My guess is that its because I let him skip his quiet time today, and now he is a bit overtired. In all likelihood he will wake up earlier than usual tomorrow as a result of getting just a bit overtired! It is frustrating that we can't be more flexible with him but now that I understand the patterns, at least its not a mystery and I can make informed decisions about letting him skip his rest time, stay up late for special occasions, etc.
So the keys for us have been early bedtime and, when he gets into a pattern of this for a few days, melatonin. It is incredible how little melatonin it takes for him to fall asleep easily and quickly, which tells me that his brain sometimes needs that little "bandaid."
The early bedtime part is HUGE. If I have any reason to believe he is a bit overtired, he goes to bed BY 7pm, sometimes we have the light off and are singing songs by 6:30 if he's really overtired. When I am diligent about keeping bedtime early he sleeps later in the morning, he rests more easily during the day (by this I mean, even if he doesn't sleep, he plays quietly and happily; if he is overtired, he will either be trying to stimulate himself by jumping around, yelling, or he'll stare at the visual timer we use to count down his quiet time), and he falls asleep more easily at night.
If you want to PM me, feel free! I've dealt with this a lot and have really spent a ton of time and energy trying to understand and manage it; I totally know the feeling of watching your child stagger through his day exhausted and knowing bedtime, which should be a welcome relief, will be a battle. I would buy some alcohol-free melatonin and move bedtime up and see what happens.
I disagree with those who say that you should follow his lead and not stress about this - it is your job as his parent to be sure he is getting his basic needs met and one of them is sleep. Some kids do fine on their own, good sleep patterns come naturally (my younger one, thank the Lord, is this way), but for the other type of kid, you have to regulate it for them at this young age. So you are doing the right thing looking for ways to work on this.
Keep up the good work and good luck!