You don't mention how long you have been married to him, couple of years? Does he go back home to visit his kids? Privacy is always good, but so is being honest and upfront and something doesn't feel upfront about this.
I would say these are all things you should have learned about and asked prior to your marriage with him, but you are married now, so you will just have to give it time and continue to share your feelings.
My husband had a previous marriage and he still had a framed wallet sized photo of her in the other room. I didn't mind it being there when we were in the courting stages, but once he told me he loved me, I didn't respond to it and we had a talk. He had already said he was divorced, so I asked him if he had papers. He looked at me with the biggest eyes and asked if I wanted to see them. I said yes and went in the other room and came back and threw them at me. I flipped to the back page and found the stamp and handed them back. He told me to read them, so I leafed through them to find out if there were any children and how the bills were split. It was a pretty simple divorce. Then we talked about the need for the photo in the other room and he went in and ripped it up. I didn't mean for him to rip it up, but how could I love a man who had not moved on in his life. Many people are shocked by that move (asking for the papers), but I was not about to involve myself in a tangled weave of someone else's love affairs.
All I can say, is keep your eyes open and it seems you have and continue to tell him what you want. I hope all goes well for you.