First, don't fear bringing him in because it's a nice restaurant. If crowds and noise bother him, yes, leave him with grandparents.
Nosy people will just have to learn to deal(my opinion, from experience with nursing in public).
To prevent yourself from becoming engorged, you do need to have him take from both sides, or pump if he's full after one side, and they feel equally soft.
Since he feeds on demand, you want to pump as much as possible, so he isn't hungry.
Since you have time, see if your parents will take him for a couple of hours(about the time it takes to have a nice bubble bath and taking care of your hair), and see how much he eats then(use the smaller bottles to measure), and then multiply x 4, and pump that amount for the day.
How often do you pump? That seems like a small amount of expression. I never failed to pump at least 4 oz at a time. If you are struggling to express that amount, you may wish to pump more frequently to learn how to pump more efficiently. Also, place a warm washcloth over your breasts for a few minutes before you start pumping to get them ready.
Hope this helps.