Did he act like this before you married and moved in? If so...why did you marry him? I'm truly not asking this to be mean, I'm really not! However, it's something you need to ask yourself. What is it you actually love about him? How does he treat you? Is he kind to you? Is he helpful around the house? Does he encourage you and make you laugh? Do you enjoy being around him? Was he convenient? Did he fill a void? Was he good enough, at the time? I have a feeling when you begin to ask yourself why you married him and what you like, or don't like about him...you might find you didn't marry him for the best reasons. I'm not saying this to harp on you, but I'm just trying to be honest and help you.
A husband should NOT come first, when he is mistreating a child. You are your child's advocate. He is cruel to her and nothing is good enough. She will be harmed more by being around him, then being taken away from him. I have a feeling she would be more happy, actually. If she is arguing with him she does NOT have respect and regard for him. And...he doesn't deserve it. I'm not normally someone who advocates divorce...but this relationship is not healthy and he does not want a child. This much is obvious. You and your child deserve better then a fill in man...which is what he sounds like. You clearly love your daughter and want the best for her. I think you already know what to do, and I hope everyone here confirms it, and gives you confidence to move forward without him.