Even though your pediatrician has said not to worry about it, I would pursue having your child evaluated for Speech issues. My middle son was slow to speak and I brought it up to my pediatrician several times especially at his two year old check up. By 2 and 3 months, I decided to have him evaluated anyway (you can go through the state's regional center which is free regardless of income for children under 3)
Also once he is three, you can go to one of the local universities in the area and see if they have Speech programs. San Jose State has a program because they need their Speech Therapist students to work with real children. The program is wonderful because they are watched the entire time by their professors to make sure they are giving your child the best therapy. The therapy is less expensive than going to a private office.
My son qualified for 2 hours of speech and 2 hours of OT a week, even though my pediatrician said he was fine and not to worry. He is now seven and above grade level in math and reading so being slow to learn to talk does not mean that there are signs of further learning problems.
You can call your local school district and ask for the Early childhood Regional center phone numbers. (I know that there is one in San Jose because that is where I went) but there is probably one closer to you.
I think it is important to get the language started. As you already suspect he is frusterated and I would be too if I could not communicate effectively.
He also sounds bored. My 11 month old daughter is a much happier child when we are out and about especially if it is a child centered activity. Of course you can't always be away from home but if you can schedule some park / library / beach/ museum/ playdate time so that he can get exhausted maybe he won't get into so much.
Good luck!