Your baby is not quite 4 months, right?
So you've still got hormones going on, and fatherhood might be freaking your husband out a bit (tiny babies to the darnedest things to men)?
Give yourselves time to adjust to being parents, and being a couple with a kid. Some people realign their relationship smoothly, with love and grace; some of us aren't smooth, loving or graceful to our partner at first. We're freaking out because here's this tiny baby and our partner is doing it wrong!
If you have help with the baby (parents, it looks like in your profile), maybe you can have a little couple time each week? A movie, food, even just a walk.
Also, not sure if this is a factor for you two, but if you haven't plopped the baby in his hands and walked away for a couple of hours (visit friends, shopping, nap, run, whatever), DO IT. And keep doing it on a regular basis. Sometimes the backseat drivers need a turn driving, just to give them perspective.
Unless the "yelling" is serious or there is any hint of violence, don't make any rash decisions right now if I'm right about the baby's age. Hormones play with your brain for at least 6 months.