He's not "too old" unless he wants to be too old. My husband was 48 when our daughter was born (my only) and I was 41. I let him put off having a baby for every reason under the sun (but mostly the stress of dealing with his ex and his two kids and three lawyers, two mediators, several custody evaluations, and then raising his kids) for 13 years of marriage and then he said he was "too old" and I put my foot down.
We did go to our counselor to discuss this and she totally took my side. She looked him in the eye and said "you WILL do this for her after all she's done for you and your kids. Think of this as a gift to the two of you."
Luckily, we were able to do it the natural way. Our daughter is the brightest part of both of our lives. Neither of us want to think about what life would be like without her. She's 9 now.
I suggest going to a counselor to get these feelings out. Men have emotion about babies, too. Your husband's reasons aren't "silly" but he may have difficulty explaining them to you. Men tend to be scared that they won't be able to "provide" for them like a man is supposed to. Age is a factor in these feelings.