An in-person apology and handing back of the goods to the other child in front of the teacher or principal can be a terrific lesson.
I stole a small red stapler when I was in fourth grade. My mom asked me about it (I lied and said a friend had given it to me), then figured that it was a lie and totally busted me by having the principal call me and the other kid out to the principal's office. I had to return the stapler to the other kid and apologize. Just the humiliation of the whole ordeal was enough to convince me not to ever do it again. I should add that I was already grounded, but I don't remember other punishment (besides a spanking for lying)--other than the total and complete humiliation and being labeled a thief within that classroom. That was huge.
I'd say that if you go too far, your son will be mad at you and not at himself. There are already social implications for him between himself and the other child. I believe the letter of apology is fine, but don't let that substitute for apologizing in person. When he's at a point where he can listen, I'd also make sure he understands the worst part of stealing-- it can damage trust and relationships between people.