J.X.
As a Walker Art Center member, I say shame on anyone who glared at you. Walker has made a great effort to include young people in their programming decisions. They have a youth board that selects certain events and exhibits for the museum. As an institution they realize how important it is to expose the next generation to the arts. I would encourage you to take your children back anytime you feel like it. There are usually more children there if you go over the weekend or on Free Thursdays. We take our active 14 month old son to any museum we feel like visiting.
I love the MIA, but it might be better for a slightly shorter visit with little kids. My son loves the sculpture, so that's what we focused on. Your older kids might think the Asian wing is really cool. They have reproduced buildings, horse statues, kimonos, etc. Also, admission is free every day. www.artsmia.org
The Weisman Art Museum on the University of MN campus would be another great option. It's not too big, and I seriously doubt the students would be that uptight about the kids. FYI, they don't have a cafe to stop at for a snack. http://www.weisman.umn.edu/
One more thing -- you can go to the public library and check out a pass that will give you free admission for 4 people to all sorts of museums. I believe it is sponsored by Macy's.
I forgot this when I wrote earlier, but the museum in Minnetonka that you've heard about is probably the Minnetonka Center for the Arts in Orono. http://www.minnetonkaarts.org/ It is more of a place to take affordable classes, but they do have a gallery. It's a cool building, and might be a nice place to visit in the summer on your way to a beach on Lake Minnetonka.