I am sorry - I am a little late chiming in, but your letter struck a chord in me. M., if you have a loving husband who wants to adopt your son and be a real father to him, then just let the biological father go. All that matters is your son and and a disinterested father is not the best option.
I tried for years to get my son's father to do more than just pay the court ordered child support and the result of that is a 13 year old son in therapy with a chip on his shoulder and anger at the whole world. Hindsight, bring 20/20, it would have been in the best interest of my son for me to have let his dad go and not tried to create a bond that the man just did not want.
You are blessed to have husband that sees past biology and wants your son to be his son also.
I wish you the best, Leyahn