Reading your request, it sounds to me that you would very much like at least one more child, but that it's not the best time right now and you're afraid if you wait it would be too late... and that you're trying to reconcile your fears of waiting.
Which is totally valid. You could wait and not be able to conceive, or worse. You could wait and have no problems at all. I think it shows a very thoughtful and mature approach that you're trying to see if you would be able to mentally/emotionally handle the worst case scenarios.
Obviously you're the only one who will either "know" or be able to reconcile. Here's my experience however:
1) Every decision I've ever made based on fear (except flight/fight type...I'm talking a life decision based on fear)... has turned out very very badly.
2) I would LOVE to have a huge family... between 5-10 kids. Big, loud, boisterous... would absolutely love it. Twins, triplets, singles... bring on the fun. (I too come from a large family -oldest child-, and have friends with large families, and simply love the lifestyle.)
I can't. I shouldn't, because pregnancy hormones trigger a cancer response in my body, treatable but painful and dangerous... and I won't because my husband is not a kind man.
What I've had to do is reconcile my actual life with what I want. I've had to decide what is important to me *right now*, realize that I have at least a modicum of control over the situation (I could, for example, leave my husband) and that a lot of life is simply based on luck and how we deal with what life throws at us.
If I never have any more children, I may be wistful (can't guarantee grandchildren either! I'm the only one of my siblings who wants children...so there aren't even nieces and nephews on my side... and my mum who could reasonably have expected a LOT of grandchildren only has one)... but it won't destroy me. There may come a time in my life where I choose to have more children, either biologically or adopted. There may not. And I'm okay with that. Some people wouldn't be... but I am.
I think we all reach a point where we're happy. Either with who we already have in our lives, or the potential of others, or both. I know moms who are "done!" at one or two or three. I know moms who keep having children until menopause (because they want to), & I know moms who keep having children even though they don't want any more.
Whatever decision you make WILL be the right one. To wait, not to wait, or to be done. You'll find the answer that makes you feel right in your heart... because you're LOOKING for the answer. You'll find it.
:) R