My first thought is he's going over there so he can sleep, not be woken up by you and the kiddo as you start your day. He works late so he needs to sleep late. You have to respect that he needs his rest.
Your spidey senses are telling you that you need this year because something isn't right. You know in your subconscious that this guy isn't right, not right now. He is either really immature and needs this year to grow some or you are too demanding of his time or something so he goes over there to get away. Not trying to be mean.
My first thought was he probably needs to sleep and you are up and getting kiddo up and ready to go to child care. He needs his sleep, especially if he's working that later shift.
When I worked 6pm-2am I came home, ate some dinner, watched TV for a bit, then went to bed. I slept until about 2 or 3 then went and picked up school kids. Hubby came home and I went to work.
Even when I was off work I kept to my work schedule for sleep times. Our bodies should be kept on the same cycle as much as possible.
I have a friend who works the night shift in a group home. On her days off she tries to stay up all day so she can do stuff with her kids and then when it's time to go to bed that night she is so exhausted. She will do this the next day too. By the time she is to go back to work her biorhythm is so messed up that she is like a zombie and every time she sits down she falls asleep. Her health is suffering too. She had a heart attack due to the stress she was putting on her system by not going to bed around the same time each day and getting up around the same time every day. If you work day then sleep nights. If you work evenings then sleep in in the mornings. If you work nights you sleep all day, every day, off work, on workdays too.
If you want someone to come home the same time you do and go to bed the same time you do and gets up the same time you do so you can spend all your time together then you need him to get a different day time job or you need to switch to an evening job or you both need to figure out how your different jobs can work together.