I may not be the best person to answer your question, as I have never owned a minivan... I did drive one once though :) (dealer vehicle while service done on my own SUV).
I think one reason (aside from vanity) that I prefer the crossover/SUV is the handling and visibility. I may be WAY off (like I said... it is speculation on my part here) but it seems like a minivan is gonna drive like.. well.. a minivan. I prefer to be up a little higher for visibility in traffic (so I can see past the honking SUV next to me) but also be able to maneuver the vehicle into tight parking spaces, etc. Also.. I haven never liked the window/brace configurations... I felt less comfortable in that minivan about backing up or changing lanes.. because I worried I didn't see something/another car from a blind spot.
And the one I drove didn't have the acceleration and "pick up" that my car does.... something that is important to me b/c I drive interstate daily... and maneuvering in/out of traffic/lanes/exit-on ramps requires the ability to "gun it" and actually GO IMMEDIATELY.
I currently drive an Xterra, and LOVE it. (I only have 2 kids, though... lol). I can see EVERYTHING... very tiny tiny blind spot that is easily remedied with the side view mirror's proper placement/angling. And I can park the thing ANYWHERE I want to. It has a very small turning angle... so a 3 pt turn isn't a 5 point turn.. it's a 3 pt turn like it should be. It is actually pretty fun to drive. My hubby has a Mazda 6, and while it IS "zoom zoom" fun.. it has a crappy turning radius compared to my car. We ALWAYS take my car as the "family" vehicle. My husband is about the only one ever in his car.
Now, again... I am speculating about what the current mini-vans' capabilities are these days... but from what I experienced... they just don't have it when it comes to pick-up/acceleration, visibility around you, and maneuverability. Not all the Crossover/SUV's do either.. but those are my big issues.