I agree with Ina... it doesn't matter if your baby is breastfed or formula fed, as long as he's fed nad pooping, the jaundice will go away.
My youngest was severely jaundiced, his highest number was 22. My doctor never once mentioned formula, just told me to feed him often and feed him until he is done. my oldest wasn't jaundiced.
Make sure your baby is getting lots of hindmilk, don't switch breasts mid feed, let him eat on one breast only per feeding. That way he gets the hind milk, that has all the 'kick'.
My son's jaundice peaked on his 11th day, and then went down, but he was yellow for a good month while his body worked out the rest of it.
Just stand your ground, refuse formula. They cannot force you to give him formula, and it won't do any good and likely have a reaction like your oldest son did... and its just not the best idea, his body is already working hard to flush the jaundice.. why make it work harder to digest formula when breastmilk is JUST what he needs. :)