I would definitely talk to the doctor.
That said, my husband had a problem while finding the right Adderall dose where in the middle of messing with the dosages there was a stage where the meds made him much more aware of his problems, limitations, frustrations and pattern and yet did extremely little to correct them.
His family is prone to anger problems, and this middle stage was just so humiliating and frustrating to him that he went through pretty bad rages with himself and everything around him until the doc let us up he prescription.
It might be that coming off the meds is crashing him so hard that he's hitting a similar sort of frustration wall where he just emotionally can't handle how incredibly difficult and painful things are becoming off the meds and how much the exhaustion is kicking in.
Sometimes changing the exact timing of taking the meds can help some with that. Maybe he could start it a little later in the morning since most of us are less tired in the mornings, and thus get an extra half hour or hour at night before he crashes which would help him get past a stressful period for him before he's without the help.
Sometimes it also might help to start some soothing rituals if he is at all amenable to that, shortly before the meds are supposed to start falling off for him. Rubbing his back or other things which stimulate positive brain chemistry might help him ease between these two times with less hard crash.