Many husbands struggle with accountability...many men are trained by their mothers to be OK with a little white lie here and there, while many girls are trained to be "good," which means responsible and trustworthy. It sucks. I've been there, I know.
What worked for me was an ultimatum. Your man needs to realize that he cannot walk all over you by lying about you and your relationship, and thinking he can slide on through this marriage by just being absent and lazy. You are worthy and valuable, and you contribute tons to the relationship just by being a mother! When I got into a similar rift with my husband, I told him he needs to be present in our relationship or I am calling a lawyer and getting a divorce. Once before, after over a year of me stating that we were in trouble, I sought a counselor and he refused to attend with me, repeatedly, until I stopped going to. Our problems escalated, and I had had enough. This ultimatum scared him into shaping up. I told me that I would not be seeking a second counselor, since I did it the first time and he arrogantly refused to come with me. He got us a second counselor and we went. Things are not perfect, but they are getting better. It made me realize that I was wasting too much time of the illusion or the "supposed to be" of my marriage, and not looking at the real picture. Sure, for all of us, marriage is "supposed to be" about equal partership, fun, love, respect, and mutual support. But mine was not, and I was refusing to accept that. When I accepted that, I was able to make a clear-minded and reasonable request of my husband. It was harsh, but it had to be done. I am young woman and I don't want to waste my life away with a man who does not even care about me. I am sure that you don't want that either.
I say, be bold, and make that ultimatum. If he loves you, he'll get it together. On the other hand, if he really doesn't care, that's OK...gather up your little ones and move on. You'll find a man who will care for you one day.