The only way to do this legally is for your ex-husband to give up his parental rights. Unlikely, I would imagine. But the legalities don't have to dictate what is the best decision, necessarily.
When I got remarried, I wanted the same thing - for my new husband to raise my older children, and maintain the household in the event of my death. He was willing, and we put that intention into the will. However, we knew that legally, my ex would still have 'dibs'. Putting it into the will was just a way to make my intentions clear, and to show that my new husband was willing and able to maintain this responsibility in the event of my death. We made my ex-h aware of this, but did not ask him to give up his legal rights. We said in a letter, that we just wanted him to know about it, and that if anything ever 'happened' to me, we would sort things out then in the best interests of the children.
Unless your health is in jeopardy, or you habitually do risky things like drive drunk or use drugs or base jump, the likelihood of you dying in the next 10 year is probably pretty small.
If you did die, the courts would grant custody to your husband. However, if time has passed, and relations have stayed cordial, he may not see fit to 'sue' to get his daughter back. If he respects your husband, and sees that the home situation is good, he may realize that the best thing would be to meet and make arrangements to let her finish out her childhood years with your new husband. He could visit her more, and do right by her without disrupting her life even more. Also, years down the road, his own situation may preclude him suing for full custody. And also, once she is a bit older, she may have some say in the matter, and her wishes should be considered.
Best of luck in sorting out what is best for your daughter and your family. In my case, we have made it to where my kids from my first marriage are now 19 and 18, with no lawyers in sight. :)