I don't know if he was that small, but I have a relative who is a pediatric dietician who has a tiny child. Her child eventually grew in middle school and although slim, is now a typical sized adult.
She used to just shove healthy snacks into him constantly. I don't think she resorted to Boost or Ensure or anything like that - but grapes, raisins, etc. were always in her purse, and any chance she got, she fed him. He was just very small.
The thing was, if you looked a pictures of his father - he was a tiny child in his school pictures. So is any of this hereditary?
Just like some kids are off the charts naturally for weight and height (mine were all in the top percentiles at that age and are now mid range - happened without me changing anything in their diets) - some are small.
They tend to normalize I guess during the school years. If they have performed tests and he's seen the proper doctors - I would just let it go. I'm not sure why they felt you were starving your child and had social services involved. That seems rather extreme. I haven't heard of that before. Sometimes they will intervene if they think there's a problem with being able to afford nutritious foods. However you haven't mentioned that was the case.
I wish you the best and hopefully things work out. Keep us posted. Your story and progress will encourage other families :)