M.D.
There is a great book called The Growth Mindset (Carol Dweck). It's not just a parenting book - in fact, most of it is not about parenting. But there is a chapter that is really good parenting advice. In a nutshell, it talks about a common mistake parents make - praising good performance instead of effort. When a child comes home with a good grade, parents say great job! But if a child comes home with a not-so-good grade on a project she/he worked hard on, we don't give as much praise. Over time, they internalize the idea that the goal of life is to get things right (perfection), which sets them up to be afraid to try new things because they fear trying and possibly failing.
The book explains the concept much better than I can, and it's available in libraries so I encourage you to read it. It might give you some good ideas about how to interact with your child to make sure that you are rewarding effort and not perfection. It's not easy - I still struggle to do it right - but I think it's important so I keep trying. I think this could really help your child's confidence.
ETA: 2 more thoughts specific to reading: If your kid likes TV and you let him watch a little, get some Super Why DVDs and let him watch them during his TV time. It's a PBS Kids show. My kid loved that show and it's really good at teaching kids how to sound out words, etc. Second is that I agree that you should let him pick up what he wants to read. My kid knew how to read but had no interest until he found the Captain Underpants books. They are graphic novels and all the pictures really help a new reader to get into the storyline, and my son loved the silliness of it.