B.,
There are soooo many thoughtful answers to your post already here, I'm not sure if I can come up with anything original! I guess my gut advice would be, follow your own heart. You & your hubby hash it all out, and, yes, even make a "pluses & minuses chart" to every factor you can think of & all the very insightful ones laid out here. Don't be afraid to think of e-v-e-r-y-t-h-i-n-g including the unthinkables like, "what if something happens to (me or hubby), would (he or I) be able to take care of our bigger family alone?" Once you've made your big old list & pondered it a while, sit back & listen for the quiet voice inside your heart to guide you toward a decision.
IMHO, you are not "too old" at all to contemplate/have another baby, but like others have pointed out, there are some variables us "oldsters" (I'm 45 & having the same internal conversation) have to think about...like difficulty conceiving, possibility of birth defects, how old we'll be when the baby is entering college, etc., etc. It is such a life-changing and personal decision, and one that no one else can make for you & your spouse.
Might I also suggest the possibility of adoption, or fostering-to-adopt? You sound like you have a tremendous amount of love in your heart that a child would be so very blessed to share. We see on the news all the time about how many children there are in the world who don't have loving families to guide them & help them grow. There may be a child out there, somewhere, whose "puzzle piece" fits the shape of the empty space in your heart. You might talk about that possibility with your hubby & see if that's a road you both want to explore.
Like I said, the answer is in your own hearts & is one that only you & your family can make. Whatever your decision, I wish you & your family peace & light during this holy season!