L.
I went through this with my son when he was very young. I found another sitter, even though there weren't any sitters in that town, I asked around and an awesome gal offered to watch him, whom he loved. The problem at the first sitter's was that she had all the kids lay on mats to take naps. My son had always been read to or rocked to sleep, and was lost and scared to follow her protocol. She didn't understand or have time to deal with that. Later when he was school age he went back to that sitter and was fine.
The reason I would not take him back when he screamed and cried was because I didn't listen and respond to my two older children. I should have and regret that I didn't listen to them because it was a bad situation.