He may "love" you, but along with love comes respect, and he is not showing you any.
It's not too much for you to ask for him to go out with you, just the two of you once in a while for dinner and a movie. Unless your family are a bunch of insufferable idiots, it'snot too much to ask for him to spend an hour once in a while making small talk with them over a cup of coffee.
The kids are his too, and he needs to man up and take responsibility for some of their care and upbringing.
He walks on the floors, he should mop them some of the time. He eats, he should cook and wash dishes some of the time. He uses the toilet and tub, he should scrub the bathrooms some of the time.
Personally, I could not live with a man who behaved that way. I would be to the point of telling him that if I'm going to be doing all the work of a single parent, I'd just as soon BE a single parent. But that's just me.