Your son sound a lot like mine in some regards. He was always the same with food. He is now almost 6yo. I don't think he said the words, "I'm hungry" until he was four or so, even though he could speak very well by the time he turned two. He still doesn't "like" eating and I don't know if he ever will. He does get hungry often, but would put it off for just about anything. He has always been around the 50th percentile for weight, but a lot of that had to do with the fact that he was very tall for his age.
One thing that helped us a lot was getting him on Prevacid when he was 3.5yo. He ate much better after that and when we took him off of it at one point, he stopped eating again. I know you've been to a GI, but have they recommended or tried Prevacid for him? I don't think it would hurt to try. He's still on it and will probably have to be on it for a long time. He even had an endoscopy and a bunch of other tests and they never found anything, but we've kept him on it anyway.
Also, I rarely gave my non-eater snacks to get his appetite up good for larger meals. I think that's also good for fat reserves. You know, if you want to lose fat, you eat small portions more often. The opposite holds true.
I now have a baby (also 15 months) who is the opposite extreme. He wants to eat _all_ the time. I've had a hard time typing this because he's begging me to feed him. He eats so very much and I can't keep up. It is very nice to have a good eater, but it seems that feeding him takes up so much time because it's constant. I think my baby eats as much as my almost 6yo at this point. It's kinda crazy.
No, you're not crazy. You're just a concerned mom and we've all been there. Just keep doing your best, like you're doing and always be an advocate for your child. It's very hard sometimes, but you'll figure it all out eventually. My non-eater does have some other issues (you mentioned that yours is a "child of extremes"). He most likely has Sensory Processing Disorder (or Sensory Integration Disorder). Always an extreme personality. I feel for you, really I do. Just enjoy him and be positive.