Real estate is a nasty business all around, which is not to say that you should also be nasty, but definitely don't be afraid to stand completely up for yourselves.
It's possible that since you haven't been that serious about buying a house in the past, that he doesn't realize how serious you are now, and doesn't want to do a lot of work for a low chance of payoff. On the other hand, if you made an offer, that's pretty obviously a good sign of commitment. Definitely talk to him and see if you can get a feel for whether he'll step it up now that you're really serious.
My sister switched realtors, but that was pretty early on; their realtor sent them very few listings and refused to show any houses after dark (basically that meant they could never go after work). So they switched after a couple weeks. Their new realtor was great at first but got a little questionable during negotiations on inspection items.
Also I'm sure you already have an attorney (if you don't, just stop reading right here and go hire one); once you've made an offer and especially after the inspections are done, it's a great time to let your atty handle things. Realtors often collude with each other to the detriment of their clients (staying barely within the law, or at least hard to catch); the lawyer is the only one working definitely for you. I've had lots of issues that dragged on and on and nearly ruined deals, that the lawyers were able to work out in a couple days.