I hate to agree with your husband on a subject this touchy, but it would certainly seem that sometthing is being said while she is at your grandmothers. Otherwise, I would think she would act that way more often and in other circumstances as well. I wish I knew what to tell you. I'm sure it's happened before, if that's any consolation. I have a friend whose mother hates her husband. She says things about him to almost everyone. However, I don't know that she has ever said anything directly to their child. Maybe she has and he just isn't big enough to realize and act on it.
Ask your daughter why she's behaving that way. I wouldn't ask her directly if the grandmother is planting something, but, if she refuses to talk, I might mention that it only happens at certain times. I would talk to her without your husband first. If she confirms the grandmother is saying things, speak to your grandmother immediately. I would try not to be accusatory, remain level headed (even if you want to blow) and, again, I would certainly go alone. Taking your husband will only add fuel to the flame.
Assuming it is the grandmother, I doubt there is going to be a simple solution . Grandmother is probably going to get very upset when confronted, but she has to made to understand that is not acceptable and action will be taken (ie, supervised or restricted visits) if it continues.
Of course, if there happens to be another reason for this behavior, that needs to be addressed as well. It isn't good for anyone when a child is hesitant to go to a family member. This is especially true when a child is afraid of a parent. I can only imagine how upsetting it is for him when she refuses to go to him.
