I agree that you do need some time away alone. Have you sat down with him -- away from home, and where there are no kids to interrupt, and at a time when he is not stressed from work or nearly out the door to somewhere else -- and told him, this is important for our marriage and without us having a strong marriage, we cannot both be strong for our kids? Put it in those terms and see how it works.
If there is a larger issue, consider whether you and he need marriage counseling. If he seems to be using the children as an excuse not to spend time with you, rather than wanting to be with the children because he adores them, then that's an issue. You and he need better communication, which you seem to recognize, but he may not. It's time to get going now, even with the help of a counselor, because frankly you sound as if you're....well, somewhat resenting your own kids a bit, because he prefers time with them. Even if that's not exactly true, that's what's coming across.
One other thing: Take a deep breath and look at all the Mamapedia posts from women whose husbands do nothing with the kids, or have to be begged into doing anything with the kids, or who are utterly clueless and uncreative about finding things to do with the kids. While your husband clearly should give you more attention, think about those posts, and how that is a much worse situation than having a husband who is interested in his own children enough that he spends actual time with them. I am not trying to diminish your feelings of rejection-- they're real, they're yours, and he needs to know what you said to us here -- but just asking for a reality check: So many dads may be loving but clueless and detached, when you have one who is at least erring in the other direction -- involved and caring.
One last thought -- it's very, very typical that before marriage the men do the wooing and planning, but afterward, the women do the wooing and planning in the relationship. It's not great, it's not really right, but it is typical. So you may have to set up all the babysitting and plan that just-you-two getaway yourself to pull him away from the kids long enough that he realizes he does indeed like that time with you.