He is absolutely not getting enough nutrients - milk doesn't begin to have what he needs. Actually it's not even a necessary food. I agree about giving him some control if you can put him in a booster seat or something else less confining. Give him some foods he can eat with his own fingers and you can even try giving him a fork he can manage himself. Get a fun dish/utensil set. Put a toy in front of him - one of those suction things that can't be pushed around.
If he just plays with the food, take him down from the seat and send him on his way. When he is hungry, put him back at the table and put the same variety of foods out in front of him, letting him choose what to eat. Don't give him more milk than you normally would, and consider cutting down.
There is a fantastic kids' supplement called Now for Kids which will give him tons of great nutrients in a balanced blend - it can go in his milk, juice, applesauce, yogurt or water. It comes in vanilla and chocolate and is delicious. The vanilla can go in juice and applesauce, and the chocolate is great in milk & water. I just came from a conference and listened to numerous MDs talking about how they use this in their practices here in the US and in 3rd world countries where it is literally saving children's lives. It is highly absorbable and therefore a tremendous value. If you want more info, let me know.