My husband was a total baby about his vasectomy, too. He didn't listen to the doctor and ice the area right away like we was told to because "it's fine" and then he swelled up to the size of a grapefruit. Not even kidding. So then bruising ensued, and it took probably 3 weeks for him to feel okay again. Note that I did not say "normal" because I don't even care how high of a pain threshold a man thinks he has, he is going to whine and complain like crazy about a vasectomy. Guaranteed. ;) I did point out to him that I didn't complain nearly as much about pushing two 8-pound humans out of my girl parts, and the subsequent episiotomy/tearing stitches; he was unimpressed. Evidently the three tiny incisions on "the boys" were FAR worse. LOL His doctor DID tell him that some men have an extra-tough layer of muscle (or something?) down there, and for those men, they are going to be in more pain afterward. So your husband and mine must be in that lucky club.
Anyway, unless there is heat or oozing, or he's running a fever, he is probably fine.