C.M.
Hun, you are in shady water going in without an attorney. If you can get one- do it.
Divorce is an ugly affair and I have no doubt he is trying to 'break' you. This is a classic attorney move, prolonging it. Think about it- every time the attorney speaks to you or him, has his secretary draw up papers, etc; HE MAKES MONEY!
If he has made threats to not return the child- document it. That is a valid excuse to not allow visits.
For the child's sake- do neutral visits. Skype. Meet at a playground. Let him talk to the child on the phone.
And kudos to you on doing it through the court system.