My husband adopted my oldest son when he was 9 after being in his life since he was 4. My ex was much like yours, but legally he had custody of my son, even though he refused to have him even visit him and rarely called (long story). When I told him I wanted custody and child support, he gave up all rights to my son and my husband adopted him. We did go through a lawyer, and it took several months for it to finally get done and we went before a judge.
If you ex has so little to do with your son, he may be willing to sign his rights away so your husband can adopt him, especially if you remind him that he will no longer have to pay child support and that if he doesn't, you will seek to have the child support raised.
Your son is very blessed to have a father who loves him so much, regardless of if he is the biological father or not. Your family is in my prayers.
H.