My 5-y/o is also very "mouthy" this way. When he was a baby, he would crawl to the cat's food bowl and eat cat food. X-P I still frequently have to tell him not to "eat" (put his mouth on/lick) shopping carts if he's riding in them. On a regular basis, I will catch him with Legos, marbles, or other small objects in his mouth. I try not to freak out (certainly nothing to make him swallow and choke on it), but I'm also concerned that he *will* accidentally choke on these things, even though he seems to be just holding them in his mouth like some people chew gum or gnaw on a toothpick. And I'm afraid that if I draw attention to it, it will make it worse. I try to make light of it, and get him to laugh and realize it's a not-so-good habit [I say, "You're not a goat!" and we both laugh], but sometimes I wonder if it needs more than that.
Some people have suggested to me that I give him some sort of safe chew toy, but I don't know if that will work -- he seems to like small hard objects (like the aforementioned Legos, marbles, etc.), so a bigger one wouldn't seem to fit the bill. But that's a suggestion.
I'll be reading the other responses as well, to see if some of them may work with him.