The Hall of Science in Queens is the best I have found. It is huge, and most of it is geared towards kids. They also have a specific toddler area where they have great things to play with, puppet shows, storytime, arts and crafts. My 2 year old loves not only the toddler area but also many other things in the rest of the museum, e.g. the cars you can pretend to drive and the outdoor "science" playground. There is a real rocket outside the museum.
There's also a huge playground next to the museum (not part of the museum) called the "playground for all children."
The Children's Museum in Manhattan is very good, as is the Brooklyn Children's Museum. The latter has "stores" similar to what you would actually find in Brooklyn, e.g. a Mexican bakery where the kids can pretend to bake, pizzeria with Italian ice, and a Caribbean store. My 2 year old loved the very real looking city bus that kids can pretend to drive. There's a toddler area for under 5 that includes a water play area and arts & crafts.
Yesterday we went to the Steppingstone Children's Museum in Conn and were a little disappointed, it was cramped and only entertained the kids for 1 1/2 hour.