L.S.
HI D.,
I'm no expert, but I would be concerned something is happening to him that he does not like while you are gone to work. i am not trying to blame anyone for anything, but sounds very concerning. Does grandma yell, lose patience? Are the other kids being mean? I'd try to investigate deeper into it.
I hope all works out well.....