The baby is at an age where it very well could be teething, but he's also at an age where he could be having a growth spurt. At 4 months old if he's waking during the night and is actually hungry, you have a choice of things to do. You can feed him when he wakes and there's nothing wrong with that at all. Babies have very small tummies and the milk goes through their digestive systems quickly because they need it to grow. When they have a growth spurt they're burning more calories so they need the "extra" formula/breastmilk from an extra feed. You would NOT be over-feeding.
Your other option is to add an extra ounce or half ounce in each of his day bottles to see if that helps at night. Sometimes you can get all of the necessary calories in during the day before bedtime, but don't be surprised if increasing the amounts during the day doesn't work right away.
You could also try giving him one extra bottle right before bedtime, a "dream feed" and give him his additional calories that way.
If he's teething, then give him a dose of Infant Tylenol drops and if that soothes him and he sleeps better then you know that's what it was. A baby doesn't have to cry to be disturbed by teething.