You know, my husband was kind of the same way. I mean he would pick up our daughter if she cried, but mostly I just took care of her. But I think a lot of that had to do with the fact that I breastfeed our second (our son would not breastfeed) so he was able to bottle-feed our son quite a lot which I think helped them "bond" a little earlier. Plus when our daughter was an infant, I think I just naturally tended to care for her while he just naturally took over a lot of the care of our son. But now our kids are nearly 4 and 2 and he is just as close to our daughter as our son.
I think the only thing you can do is voice your concerns to your husband without "accusing" him of disliking your second son. You could also ask your husband to help out more with the baby - can you give him a bath, a bottle, and if he's crying and your husband seems obvlivous, ask if he'll hold him, etc.