Hi V.. Mom of an EXTREMELY picky 5-year old here. He has always been fairly picky but worsened as he got older (between 2 and 4 probably). He tends to eat more variety at school, so I purchase the crazy expensive school lunches for that reason. Here it's spaghetti, sausage (odd for a picky eater I think), cereal, bread, yogurt, some cheeses, PBJ, chicken nuggets, most every fruit, and SOMETIMES fries (if they are the PERFECT size, texture, etc. whatever that is). There's occasionally something else he'll eat, but he's pretty set on likes and dislikes. Potatoes are the devil, rice smells funny, beef makes him vomit, green stuff is green... sound familiar? He has a LOT of sensory problems, so texture and smell particularly are difficult challenges for him.
We supplement using vitamins, and he grows normally and has no health problems. He's a picky eater, and that's that. I have had to learn not to stress over it too much, especially because I have a 3-year old who wants to be just like big brother, so fighting over it just shows him that food is a big enough deal to fight over.
I have heard/read that a lot of kids avoid foods that they have allergies to/sensitivies to (milk, meat, eggs in particular) and that it is not necessarily a conscious thing. Their bodies just know they shouldn't eat it.
My other issue comes from the "preparing different foods for mealtime" argument that you said you got from your pediatrician. I have heard the same thing from peds, teachers, parents, friends. But my husband and I also love hot/spicy foods like buffalo wings, chili, and Mexican food which I would not expect little kids to like or eat. So I have NO problem preparing something different for them. I think it's ridiculous to expect that we not eat foods we like at our only mealtime together just because we don't want the prepare something different for them. I eat EVERYTHING and so does my husband. Our kids see that and are still picky (again, particularly the oldest). My hope is that by seeing us eat such a variety of foods, textures, flavors, they will want to sample them and learn to like them rather than us just serving baked chicken, mashed potatoes, and green beans (blah!) because they are "safe" family foods that we can all eat together (not that either of my kids would touch any of it!). You're right, he's not going to starve whether you prepare foods specially for him or not. But I know in my house I've realized giving one carrots (he does like raw carrots, although he'd rather die than eat cooked) and the other broccoli (little one loves broccoli, hates carrots) means I have both for myself (I love BOTH!), shows them that you actually CAN like both (modeling the behavior), and also shows my big boy that I'm not dismissing the idea that it may be the texture/smell/etc. that is pushing him away not a desire to just refuse the food. No stress, the kids eat, we supplement with vitamins, I know that tastes change in kids like they change their socks. Also, we have gone through phases of NutriPal bars and shakes to supplement their eating especially when there is an EXTREME pickiness phase.
Hope that helps a bit. Good luck!