My experience has been that there is no down-side to sending a young boy to kindergarten. He either does well and moves on, or he's near the bottom of the class and repeats a year. Kids that young don't care if they repeat a grade, and their classmates don't care either (unless, of course, all the parents act annoyed or embarassed and make a big deal out of it).
There's a huge potential downside to holding him back, though. If he's one of the smarter kids, he's just lost an entire year of learning and reading, and he'll forever be in a classful of students who are woefully behind him academically.
Please send him to school. Children that age are like sponges -- so ready to learn so much information -- it wouldn't be right to not expose him to reading and such, unless, of course, you're planning to teach him to read at home. (I taught all my kids at ages 3, 4 and 5, prior to kindergarten, when they were ready.)