I'll bet your son is bored. He's bright and has no patience for the kind of "training" he is getting. He would thrive if "educated"...
Schedule another 504 meeting in writing and send everything to everyone by certified, return receipt mail. Cc: everyone involved.
Insist on a one-on-one aide and special location for testing. If his teacher feels he is too distracted in the classroom, the aide can keep him on-track there too. I work as a one-on-one aide and some of the kids I've worked with are like your son. Very bright, but unable to focus.
There is NO reason for a parent to sit in the classroom to keep a child focused on their work. He needs to be introduced to stratagies that will serve him in the rest of his life. (You are not going to be sitting with him through HS and college, or going to work for him when he';s older.)
If his teacher can't offer this, then the child needs resource room with a special ed teacher, and an aide in the classroom.
Another option for him is an inclusion class. They have two co-teachers. One a mainstream teacher, one special-ed. The two teachers run the classes in tandem. It's not just the special ed kids who get the attention, the teachers help the entire class to learn in different ways. You tend to find more flexible creative teachers, with a broader range of skills in these classes.
Make a list of what you want for your son, and what the district has to offer. At the beginning of the 504 meeting, state your goals, and what you expect from the committee. Make it clear you will be recording the meeting (You are ABSOLUTELY allowed to do this in NYS) and take detailed notes. Ask for coppies of all the assessments and reports before your meeting, and ask for explanations of anything you don't understand. If they present you with paperwork at the meeting, tell them you wish to reschedule for a time after you have been able to review their doccumentation.
Also, if you disagree with the committee's findings, you can sign a letter of partial agreement, and appeal the parts of the 504 that you disagree with. When in the meeting keep the committee focused on your son and his best interests... Don't get distracted by double talk.
Also the best thing I ever did was to consult a Special Ed lawyer. For the $250 consultation fee, I got more leverage than I ever could of gotten on my own. Just being able to cc: him on the letters worked like a charm.
Two good resources are www.wrightslaw.com and www.schwablearning.com
Good luck with everything.