L.B.
While my son does not have special needs, I too required that his oft absentee father develop a relationship with him before taking him for the night or on vacation. I required a phone (heck I even provided one), a regular schedule, and adherence to my son's allergy and asthma medication regimen.
Me Ex responded by complaining that there were too many rules and that I was keeping him from seeing his son. He complained to co-workers, mutual friends, and his family. He continued to complain while I offered solutions and tried to facilitate their relationship.
Ultimately, the requirements that needed to be in place to ensure my son's well being while with his father were used by his father as excuses not to see his son. Nothing I asked was unreasonable, medication and phone - but, it turns out, he was been pressured by his family and friends to see his son, he really didn't want to see his son, and used these things as a way to excuse himself from seeing his son.
My advice is to stick to your plan. Your child's welfare is most important and having him stay with someone who does not understand, and seems unwilling to understand, his special needs and ensure his well being, no matter who that person is, should just be a no go.
Have your son's therapist write a letter outlining your son's needs and the limitations that must be placed on developing personal relationships in order to ensure not only his well being, but that the relationship can develop. Write out your son's schedule, dietary requirements, and routines. Propose the day visits, include dates. Send it all, via certified mail, to you Ex. Paper trails are good things. Put in writing that you want your son to have a relationship with his father but that he (the father) needs to be well versed in how to care for his son.
I don't know what type of child support/custody agreement you have in place. But you may want to consult your attorney also. If your Ex has court ordered visitation, you must ensure that you honor the terms of the agreement - however, if you have due cause for concern about your child's welfare when with his father you need to get advice from your attorney. It may be that you need to re-visit your agreement and have some sort of care classes included in the agreement for your Ex.
DNA does not equal responsible parenting. While I agree that all children should have the opportunity to develop relationship with both their parents, we must, as the custodial parents, ensure their well being during the process.
Good Luck
God Bless
Give your son a hug from me
EDIT: Stick with the terms of your divorce - supervised visitation until, and only if ever, you feel comfortable leaving your son alone with him. You are doing the right thing.