S.H.
have him screened by your public school district for KG readiness. & my biggest concern is that if you let him drop from his current school....to me that just sets the stage for him to be able to emotionally & physically walk away from whatever doesn't suit him in life. That's a huge issue for me!
My next concern is "why" he thinks he's dumb.....where did he hear this? Why does he think this? In other words, who's responsible for saying this horribly damaging phrase to him?
If the next door neighbors are sooo abusive, then I do understand your concerns about the public school.....but they are just a few kids....& there's a whole school full of other kids who may be more acceptable. & your neighbors are not a true reflection of what his education & possible extra help could be vs. the Catholic school he's in.
Lots to think about, but I truly believe that the new baby is factoring into the mix. OH, & now that I think about it, one of my friends just went thru something similar with her daughter. The child was miserable & very vocal about being unhappy with her new teacher. After several emails, mom & teacher met. It turns out that the child "wanted to just stay home with Mommy & didn't want to go to school". Things are much better now that the truth is out.....& the child knows that Mommy & teacher are on the same page as to her education. Hope all of this helps!