There are definite reasons to hold a child back, and reasons to not hold a child back. There is no one size fits all answer. Variations in aptitude from grades K-3 are HUGE and that's normal. A child can be ahead in an area during one grading period, then drop behind in that area when a new level of skill is introduced, and vice-versa.
My kids' elementary school uses the RTI (Response to Intervention) model for this exact reason. With regular assessments of skills (not all testing all the time, but certain assignments are given specifically to gauge mastery of an area), they can flag when something seems off track and get the child some extra support immediately instead of waiting for an education evaluation, and IEP to be implemented or changed, etc. The extra support is immediate - child is falling behind in reading fluency, he goes into an extra support reading group for 6-8 weeks and then is reevaluated to see if he needs continued small group work or not. Falling behind on addition facts, works with a helper in a small group setting to master those. And so on, and so forth. When RTI doesn't show positive results or they feel the problems are pervasive, then they move on to more formal evaluations, etc. This model works for most kids because it addresses the normal, age-appropriate variations in their learning without the dramatic, disruptive move of retention. By the end of 3rd grade, most students settle into a less erratic learning pattern.
That said, two of my younger siblings were held back, my brother in 1st grade and my sister in K. My sister was an easy case - she was born in November (December cutoff back then) and just wasn't mature enough to get through the day. My brother was actually one of the oldest kids in his class, but we were in Catholic school and it became apparent during the year that he needed SPED services that they couldn't provide so they focused the latter half of the year on testing, evaluation and placement for the next year and had him start over in 1st grade in public school. It was the right move for both of them.
While retention may be the right move for your child, I do see some catastrophic thinking in your question that may be clouding your judgment. Struggling today doesn't mean struggling tomorrow or forever. Struggling isn't a bad thing, it's where the learning is. You're right in that struggling with everything, all the time can be demotivating and damage self esteem but with the right support, he may very well turn a corner. My oldest hated school by the end of first grade, when he couldn't read, write, spell, add or subtract. His testing came back low but not low enough for services. In 2nd grade, with a better teacher and private tutor I hired for the entire school year, he was back on grade level in most areas (still on the low end, but there) and feeling much better about school. He did eventually qualify for an IEP in 4th grade and has been on it ever since (now a high school senior). School isn't easy for him, but he does enjoy it due to having received the right support. We were able to undue the damage that first grade did.
I would just encourage you to keep an open mind, and if you can afford it, maybe try to schedule a consult with an educational advocate in your area who is qualified to go over everything and weigh in on the best course of action. It can be hard to trust the school district to do what's right. but give them the benefit of the doubt and at least hear what they have to say with an open mind.