My advice is to not push yourself. Maybe you need just a little more time before you are married again, and there is nothing wrong with that. It doesn't mean you have the wrong guy, it means it might not be the right time. If you have a true bond fit for marriage, he will give you all the time in the world.
As for the kids. An "ideal" 2nd marriage is one where the spouse has no kids, and there are no more made. Before everyone gets ticked off, let me explain myself.
When both spouses have children from other marriages you often have stress from making it work with their exes. You cannot take the place of their mother or father.
When you have more children with husband #2, there is a risk that your own children will feel a little like outcasts, or jealous because you don't have the same family bond with their biological father. They might feel like they aren't loved as completely or as much.
Maybe your boyfriend understands that, and that is why it isn't an issue for him. In my opinion, I don't think your situation could be any better. (Except maybe meeting this one the first time around, but having the same children. Don't we all wish that!)