4-5 times a day is totally normal for a 9 month old. Some of it is individual too. My daughter was probably nursing 6-8 times at that age (makes 4-5 times look good, right? lol) But, my son who is now 16 months and still breastfed was probably at 3-4 times a day...maybe less at 9 months. He dropped feedings on his own. Around a year he was mostly in the morning before we got out of bed (sometimes when he wakes in the night, I bring him to bed), at naptime, and for bedtime plus anytime he woke at night. At 16 months, his schedule is pretty similar. Nap, bed, in the night if he wakes. You probably already know the bulk of a babies calories in the first year come from breast milk or formula. So your 9 month old is just fine. I don't know how great your baby is as solids, but for me it was nice knowing my kids were still getting plenty of calories from breast milk.
You are so close to 1 year that I would try to hang in there the next couple months. Then at 11-12 months you can start introducing cow's milk if you want. You might find it easier to offer him milk in a cup during your nursing times to wean him. And I'd go slow. Take his least attached nursing session and offer him the milk at that time for a week. Then drop another session, etc. Go as slow as you want. Usually nursing to sleep sessions, like bed time are the last to go.
I found 8-9 months a particularly rough time with my kids. They do change a lot in a few months time and you might find as your baby gets closer to a year that he/she nurses less on his own especially as they get really into solids and are more distractible.