After having a baby, especially if you got very big in your belly (from your uterus stretching), your muscles stretch and change. To me it sounds like you have a diastis recti, which is where the muscles separate and don't go back like they used to be. (never fear, though, there is this AWESOME book that has the perfect exercises to get it back in place).
To check and see if you have it (and the majority of women do after having a baby, especially if it was a very big baby), lay on your back on the floor with your legs bent in an upside down V. Then place your fingers at your belly button, pushing down a little bit. Then gently lift your head just a touch. Don't lift it so much that your shoulders lift off. You just barely want to lift it to barely tighten those tummy muscles so you can see where the edges of them are. Once you do that, you should feel your muscles and be able to determine if there is any sort of space in between them at around your belly button where your muscles are supposed to be. You stick however many fingers in there that you can (side by side - placing your fingers horizontally/from your side to side, not vertically/from your head to toe). If you tighten your muscles way too much, it can give an inaccurate measurement, so it's important to barely tighten those muscles.
Anyway, if you notice you have any space there, you have a diastis recti. It can easily cause you to have a permanent prego look that gets worse at the end of the day after eating. It has nothing to do with being out of shape or what you are eating. In fact, even if you do situps and crunches regularly, it won't fix the problem in most women (it actually will likely make it worse!). Losing weight will not fix it. It's just the muscles being stretched and not as firm as they were, causing a prego look.
When I had my first baby, she was only 7 lbs. I am almost 5ft 4in and started that pregnancy weighing 105 lbs (too skinny, I know!). My belly went back to normal after I had her (though I stayed around 115 lbs). Then when I had baby #2, he was 10 lbs 11 oz, and I looked prego with twins. I had a small body and HUGE freakin' belly. That definitely did a number on my muscles. I've since had a 8 lbs 7 oz baby and a 9 lb 7 oz baby, and both of those have stretched it out even more due to repeated pregnancies. I am now prego with baby #5, and it looks to be measuring on the larger side again (yippee).
But each time I have a baby, I go back to weighing around 117 lbs and my belly has a perma-pooch now. I can't seem to get rid of it, and I HATE feeling like I look three months pregnant! I have a huge space between mine - almost five fingers, which is pretty large. Most women only have 1-2 finger space (then again, most women don't have nearly 11 lb babies!).
Someone told me about this book called "Lose Your Mummy Tummy" by Tupler. It's supposed to be pretty amazing at getting the belly back in place. I was already pregnant with baby #5 when I learned about it, so I can't say it's worked great for me...yet. I am going to use it during this pregnancy though. The exercises are made safe for that. But my friend that suggested it had great results herself.
Here it is on amazon:
http://www.amazon.com/Lose-Mummy-Tummy-Julie-Tupler/dp/07...
I've just written a REALLY long response! hehe! But, that's what it sounds like to me. If you notice a space between those muscles, there's no reason to go to the dr. You just need to get this book (or one like it) and try the exercises to get your muscles back in place. It'll then be able to hold your tummy flatter and you won't have that prego look. If you don't have that space between your muscles, then no idea! Maybe bloating?