A.J.
You are doing an amazing job being a big and generous person by not being jealous or catty about your ex husband's "friend". Very few women could pull this off, and you have already scored BIG points with your kids (by their fun and lack of stress and comfort in talking to you) and the new lady and your ex by being kind.
You know in your rational, logical mind, that it is easier for her to "chip in" and HELP your husband entertain the kids for short periods of time, than it is for you to do it on your own for the majority of the time. You have a huge heart to want to try and improve and add things in. But UNDERSTAND in your heart, that you are doing enough, and your kids love you like no other human because you are their mother. There is no telling what will happen with your ex and his friend, but you will be their most important loved one forever. By being nice about the situation like you are, you are not risking alienating yourself in any way.
It's not all the way cool for him to be involving someone in his kids lives whenever he is with them while claiming she is not even a girlfriend, but, like a man, he can't do this on his own, so you are doing the right thing making sure your kids have fun.
How to handle it? The constant raves about the friend? It's not easy! I think you have to just accept that it hurts, keep a brave face, mention it kindly to your ex when you need to, be honest, but keep up the positivity and know you're doing the right thing. Kids always think new people and places are cool and talk a lot about them. I remember doing that as a kid, going on and on about how cool other people's parents or houses were... Count your blessings that you have them most of the time, and believe me, they do say nice things about you. She knows she can never compete with mom, and probably feels even less secure than you do! It's frustrating to see your ex being Mr Nice guy with her when he didn't help you out, but everyone is on their best behavior at this point, and time will tell the true story.
As for what to do with your kids, always remember, it's not what you do, but how you feel in the time you spend together. Whatever you decide to do will be great, but if it will stress you out or be too difficult or expensive, it may distract from enjoying each other's company.
I had been really busy working once, and my husband was gone a long time working, and I was driving the kids home when my daughter mentioned that we didn't drive past the cemetery in a while. The cemetery in our town is really pretty up on a hill with great view and she remembers going on walks there when she was younger. So I made an impromptu stop at Sunoco, grabbed some grape juices from the cooler and straws (big treat to them) and went and sat in the cemetery to drink them because it was a pretty day. We ended up there for over two hours, talking and playing tag and letting them run. I just resolved to have a nice time, listen to them, have eye contact and laughs and not hurry. Pretty much anything you do is good with those rules. We do weird "nothing things" all the time, if I just take a breath and make a decision to have fun with them for a few hours, even if it's at the recycling place and grocery store etc.
My friend and I went with her son once on a walk through her town. We stopped at a mechanic's shop and complimented his garage-he ended up giving us all a tour of all his equipment and cars-her son's eyes almost popped out of his head with excitement. Just relax and enjoy them in any time slot you can find, even doing free stuff and errands. As for all the work you have to do, this is real life, and they don't hold it against you. Hopefully you have their help with stuff too! That's always time well spent together as well.
You're doing the right thing taking it slow with your new guy. Now that you need someone to be good for your kids as well as yourself, your standards are very high, and there's no hurry. You have the strength on your own-all this hard work will pay off one day-keep up the awesome work!