I didn't take the time to read the other responses. I'm sure they offer great advice. As a mother of a 9 year old who has been in speech therapy since he was 3, here are my suggestions:
1) Have him evaluated by two different therapists. Children's Mercy actually misdiagnosed my son and therefore, the first 2 1/2 years of speech therapy did not address the cause of the issue.
2) Once you know for sure what is causing the problem, be diligent in doing any homework that the therapist asks you to do. It will help him to have you working with him as well as the weekly/biweekly session you will have with the therapist.
3) If you find he isn't improving, find a new therapist.
4) Never give up. My son still has issues after 6 years and I don't constantly correct him, but ask him to repeat as I didn't get it. When he was younger, after several times of the repeat request, he would throw up his hands and say forget it. I knew I had pushed once too many times then. He usually only has to repeat once now.
5) When you are working with him, look into his face. Be sure to ALWAYS speak properly around him and don't use any slang (ba ba, wa wa, paci). Ask all those that are around your son to speak clearly and crisply, too.
My pediatrician kept thinking my son was advancing so fast physically (sitting up on his own at 3 months, etc) that htey didn't worry about his speech until 3. Never can start too early.
Best of luck as I have been in your shoes and still am many years later. It is heartbreaking to not have the "perfect" child but if you start now, he will make so much progress.