I am so proud of you for taking this little boy into your heart! I think being gentle and lovingly correcting him is the perfect way to start getting the behavior that you want to see. Punishing him will only make him angry and resentful. When he says something offensive, you can repeat to him what he said in a way that you would like to hear him say it, and ask him to repeat it.
Part of his behavior is normal for a 4yr old boy too. Esp. one with so much turmoil in his little life. Just continue being firm with your boundaries, and he will begin to get the hang of it. It might be helpful for him to have transition time too to help with some of the emotional outbursts.
My 4yr old (will be 5this month) Has a Picture schedule that we use everyday. That helps tremendously he knows what is coming next. I'll put the whole day on a little plastic board. He goes to school, gets off the bus at home, lunchtime, Quite time, then we'll have some sort of activity (park, library, etc) Then home, Quiet playtime, dinner, bath, story time, bed. We taught him how to read a digital clock, so he knows when it's bathtime (7pm), and bedtime (8pm) that way he doesn't feel like he's being cheated out of playtime.