Building on the other replies, you might consider changing your response when he says 'I don't like you' to you or your husband. His statement really isn't about liking or disliking a person, but about what they are doing, so you can help him to start seeing the difference between those two things. Perhaps instead of saying 'But we still love you,' you might start saying 'Do you mean that you don't like that we need to stop playing outside and go in now?" He might still feel mad at you/him because he isn't getting what he wants right then, and so perhaps he will insist that he doesn't like you, however you can start building the habit for him of thinking in terms of people's behavior rather than the person him/herself.
I also liked Tadpole's suggestion for dealing with the other child not being as cooperative, sharing, etc. Redirecting the activity is a great way of getting past a possible conflict without letting the guest child run roughshod over your son.