A.C.
As someone else asked, are you sure they're night terrors? my oldest got them and it was more than a little freaky to be honest with you LOL
If they are night terrors I'd say it has more to do with the disruption of his normal schedule than due to allergies to the cat. Night terrors are more often attributed to stress. While the visit may be fun and exciting and all that is good that's still a type of stress on the body and mind.
Maybe try to get him to at least take a quiet time lay down (even if he won't sleep), while he's at your moms. It might calm him down enough for the rest of the day to get him through the night without the terrors.