A.S.
Here's my take on it.
He's not going to care *why* he can't go over there anymore. He just knows that he can't and he's angry. He's showing you he trusts you when he shows those emotions around you. He knows that he's safe with you. I know it can be hard to deal with when he says words that are hurtful, but keep in mind that he's really telling you that he trusts you.
Another thing is that toddlers don't deal well with change. You disrupt their world and it takes time for them to adjust, it's painful for them!
I agree with the other response that said finding a replacement, a playgroup or a consistent activity that he enjoys could help.