Are you pumping every time he gets a bottle? Are you supplementing with breastmilk or formula? Since milk production is completely supply and demand, you need to make sure you replace bottle feedings with pumpings to keep your supply up. If you continue to have problems or concerns, you should seek the advise of a professional lactation consultant, either at your hospital where you delivered, or an independent IBCLC. They can help you make sure you continue nursing while supplementing with bottles periodically.
Many babies have fussy periods, especially in the afternoon/evening around this age anyway. It's also possible he's going through a growth spurt, so extra nursings will help your milk supply catch up. Lots could be going on, you probably need some help from a breastfeeding professional. :)