I want to second what Stitch said -- get him into the Music Together program if there's one near you. It's nationwide and franchised all over the place. They don't have their own offices; they teach in places like community centers, rec centers, churches, and they are not expensive; you also used to get CDs of the music used, so you can play it all the time. Go online to find them. This is a terrific parent-and-child interactive music program perfect for his age and it's based on a lot of excellent research about how children learn music -- but to a kid it's just fun with other kids. They usually teach in sessions of maybe six to ten weeks. You do have to be there and participate -- parent involvement is vital.
He is young enough that trying regular instrument lessons is probably a sure recipe for frustration and could turn him off trying to play. Most music teachers around here won't take kids until the kids are capable of listening well, not getting too distracted, and following directions for long enough to get through a lesson -- usually elementary school age. I know one piano teacher who won't take any child younger than seven, maybe six if the child is very, very capable of focusing. Don't get caught up in programs that promise the world and say they'll have your child playing an instrument in no time at his young age. Instead, go for a fun and well-thought-out program like Music Together that keeps him interested and introduces new concepts and isn't just all nursery rhyme songs.Meanwhile, encourage what he's already doing at home.
I know there are other programs like Kindermusik but I know nothing about that one; maybe someone else can post about it. I know that my daughter loved Music Together and I appreciated the information we got about research on how kids learn music.