M.M.
This is something you really have to decide on your own. However, be advised that since you don't have any court document stating you have custody of the children, he can remove them at anytime and then file on you. He'd have them and you'd have to work on getting them back. My ex tried to intimidate me...he had throughout our whole marriage...even going as far as stating he'd burn my house down after one of our court hearings. It does get better. Right now you have a fear of him taking the kids. If he has a good job, do you really think he's going to up and leave the state? Do you really think he'd want to take care of the kids? If you have a court order for custody and support, you'd have something to fight with. Without it, he has just as much right to them as you do...with nothing saying otherwise. Good luck in what ever you decide to do.