You see... this is why they call it the terrible twos!
It sounds, from what you have said, that you are engaging in a power struggle with him. This makes kids really insecure... and can lead to the behavior getting worse, not better.
Little ones this age cannot really use logic... the other parents have who have posted have it right. Tell him you are going to wherever... tell him what you expect.
The moment he breaks the rule? Pick him up and LEAVE. NO CONVERSATION. NONE. Just say "This behavior is unacceptable, and we need to go now". Then, stop.
He will, most likely, have a tantrum. Ignore it. If he needs your help to get back in control, by all means help him... but do NOT engage in any sort of negotiation.
After just a few times, I promise you - he will get better.
The twos pass into the threes... and believe it or not ... there will come a magical day where he will be reasonable - most of the time!