I'm sorry to hear about your struggles, but I can say that I completely understand where you are coming from. It seems that no matter what mind set you go into pregnancy with, it is always hard once the baby is here.
My husband and I hardly ever fought in the 10 years we were together prior to our son's birth, but those first 5 months following his birth were full of argueing, complaining and for me (with a touch of post partum) lots of crying.
My suggestion is to first, try and stay calm in the moments of anger and if need be wait a little while and revisit the subject when the kids are in bed and the 2 of you have time to talk through what happened. Second, try and see things from each other's prespective. Both my husband and I learned that we both felt like the other person wasn't caring about our struggles being new parents -- but really we were both just so overwhelmed with the day to day tasks that we could only see what we wanted for ourselves. Talking through things really helped.
Finally, we both realized that we needed to make time just for each other -- even if that ment just an hour at the end of the day. Try turning off the tv and just talking or reading together (or to each other). Whatever you do, do it together. This is a great time to sneak in those kisses you've been missing.
And don't forget, men may not "officially" get post partum, but there is a lot of strong emotions they have to deal with having a new little one at home as well. Try and be aware that his distance may be him struggling a little emotionally as well.
Best of luck.