M.R.
I had a fibroid about the size of a plum at the top of my uterus, and i carried twins just fine. At first it was bigger than the baby bump, but the babies grew at the expected rate the whole pregnancy. They should be keeping an eye on it every time they do a manual exam or ultrasound, and if you don't think they are paying enough attention, speak up.
Congratulations!!
M