M.T.
I think you need to speak to a family lawyer about this. It also might be a good idea to esablish the custodial/visitation rights legally. I don't believe you will be prevented from leaving the country - they likely won't know at the airport that you are or are not married to the baby's father or who has custody. The issue is whether he is okay with this trip and doesn't press kidnapping charges. If he won't give a letter giving his consent, then it sounds like he isn't consenting to you taking her out of the country. Speak to a lawyer, get the custodial rules established.