Your pediatrician doesn't go by the pounds so much as they go by the percentiles, as well as their "curve". I'm not sure what they actually call this curve, but it is how the dr has tracked his growth from birth. Normal growth should go up in a nice gradual arc, with no major fluctuations. And the percentiles should be within a certain amount of each other, like the other poster said. I can't remember what they should be, but basically you want them to be close to the same percentile for both height and weight. You don't want a 90% height and a 15% weight. That would show he was underweight for his height. If his curve is good and his percentiles are close to each other, he is just a small child. My son was always in the 15-20 percentile for his age, but it was for both height and weight and his growth was steady and curved correctly. At your 2 yr appt you can ask to see his growth curve and ask about his percentiles, and tell your doctor abot your concerns. But I wouldn't be worried, the doctor would be very proactive if the numbers were off.
As for the busybodies- just tell them that his growth is right on track, he is just a small child, but perfectly healthy. It's so funny because I get the "little" comments too, but every once in a while someone will tell me how big my son is (in that tone that you know means they think he is huge and fat). I have actually laughed out loud at them, not meaning to be rude, but couldn't help it because it's so funny how one person perceives something totally different from another. They are just comparing them to what they have seen. Those people that bug you must have the big chubby babies, which to me looks totally wrong, but it's just not what I am used to. I am not going to call them fat. Don't let them bug you, they don't know better than the doctor or the mama.