I think this needs medical advice. I know some people have different experiences with each pregnancy, and I have no idea if this is predictable for you or not. But, IF doctors have reason to think your pregnancies will continue to be risky, and/or even get worse, it seems irresponsible to take that risk. The children you have will need their mom! And it's possible to conceive with IVF or reversal after a tubal if things change for you, or the medical advice changes, at some point. Again, though - I would have some discussions with a doctor and reproductive specialist that you trust and who will really sit down and talk with you and your husband.
Best wishes for an easy pregnancy and childbirth!