I'd definitely hold off on baby number two. BUT be proactive about getting some counseling. It sounds like you two need some communication skills, at least during arguments. (don't we all!) Kids can hear arguing through the wall, and sense that there is tension in the relationship, so you want to learn to resolve things quickly and with as little anger as possible, for your daughter's sake.
It is definitely not wrong to wish that your marriage lasts and you are not in and out of relationships. The best thing for everyone it to fix whatever issues are in the marriage.
Now, some men get upset about going to counseling, so I'd try to find a class on communication, and bring it up saying WE need to learn to argue better. Be sure you don't insinuate that its all his problem, etc... Let him know you love him and are committed to the marriage and want to be the best wife to him possible!
A counselor can easily do this if you can't find a class like this at a church or community center.
Best wishes!