Kindergarten readiness is not about academics. It's about socialization. Putting a shy and clingy child in a class of 22 for 5 days a week, when he didn't interact in a preschool class of, say, 12, for maybe 3 days, is fraught with problems. Did this come up with the teacher doing pre-K screening? Did it come up with the preschool teachers?
Trying to socialize a child in a 3 month summer when he wasn't socially ready after 9 months of preschool seems to be a tall order. In my experience, you can't rush it. You can enroll him in some summer programs such as town recreation programs where you leave him for the morning and see how that goes.
Otherwise, I would definitely hold him another year - you can do no wrong by doing this. Do NOT let anyone tell you that it's about intelligence or anything else. We held my son (for other reasons) and he never knew the difference.
I would stop talking about kindergarten now. Meantime, find a pre-K program for next year, 5 days a week. If there is a lunch/extended day option, choose that maybe 2 or 3 days a week to get your child used to adapting to new scenarios, people, structures. Particularly if your town has full day kindergarten, this is way too much of an adjustment for your little guy. Depending on the amount of the deposit for preschool, you could always consider sending him to kindergarten in late August if you want BUT it really doesn't sound to me like he is anywhere close to being ready.
Do yourselves a favor - give him a year. Don't over-schedule him, but give him a couple of opportunities. Perhaps soccer in the fall, an art class during the winter, and t-ball or something else creative in the spring. Our son got one organized activity a week, that was it.
If you put a lot of pressure on him this summer to accomplish everything, he'll feel the stress (and so will you) and the transition next fall will be much greater. If he doesn't do well in kindergarten and you have him repeat it, it's a much bigger problem than giving him the gift of a year to grow.
Good luck.