D.B.
He sounds like he's doing just fine and developing normally to me. Is there a possibility he could be color blind and thus colors are more of a challenge to him? Both my sons are color blind that's why I ask.
If you can video record him on a typical day to take with you to the session or before a judge that would be ideal, especially if you're concerned that he'll be too shy to speak to a therapist. However, they are trained to know shy for inability and can usually coax things out of a child. My inclination is to let him be seen by a therapist and one you both agree on and if she/he proves you right, then it looks better for you.
I'm sorry you're in the middle of all this. BTW if he can recognize his numbers, the color blind test uses a method where they have a tablet or device and they are asked to put their finger on the number and trace it. If they are color blind they most likely can not see the number. There are some online things too but the ones in a doctors office are far more reliable.