Can you elaborate on this a bit, and explain why you say he is lactose intolerant and unable to have yogurt or cheese, but you are putting him on cow's milk?
I think you may be switching around much too much, and he's not getting a chance to get used to anything. Also, he's not eating much table food, so I'm wondering if there are some sensory issues that make it hard for him. A lot of kids don't do well at this age with texture, so a variety of table foods with a different "feel" in his mouth can be problematic.
I work in food science with a lot of kids like this, and in fact with kids with what appear to be healthy appetites. The AMA said, over a decade ago, that the American diet (even with whole foods) is insufficient for full nutrition, so everyone MUST supplement. I would think a truly comprehensive children's supplement with all the necessary ingredients for healthy cell functioning would be highly beneficial. We do this all the time with kids who have compromised immune systems, malnutrition/failure to thrive syndrome, and more. We use a non-dairy and gluten free supplement that can be drunk by itself or mixed in many other foods the child will consume. One of the interesting things that happens with kids (and adults) with sensory issues is that the problems often go away when the body gets the fuel it needs. Many of these conditions are the result of nutrient deficiencies, so if you give enough in a highly absorbable form, things can turn around pretty quickly, especially with advances in epigenetics which controls the way the cells function.
You're right, he needs more calories and more overall nutrition. Even if he drank milk, it's not a complete food. So you need more than that.