If it is at all possible, please get counseling for your grandson, daughter, and son-in-law. Many counseling offices offer a reduced rate based on your income if that is an issue. Look at it from your grandson's view: Stepfather loves his son, but not me. Now mom loves Stepfather and Stepbrother more than me. She's sent me away so she can create a new family without me. If you don't think that's what he's thinking, you're in serious denial.
Others are right, and I imagine you know this-- when you become a mother, EVERYONE else is second in importance to that child. Including yourself AND your husband (whether or not he's your child's father). It is our job as parents to care for our children. That includes finding out why our children are "misbehaving" and help them out of that pattern of behavior.
Your grandson needs to be integrated back into his family. A counselor will be able to help them find the best way to do this. If Stepdad isn't willing to do this, then Mother needs to make the decision to leave. Please help her see that her child needs to come first.
In the short term, while he is with you, just stick to your rules. It sounds like what he most needs is consistency. If possible, sit down with him and come up with a list of house rules TOGETHER. Also come up with a list of possible consequences for breaking the rules. They should be of varying degrees, from losing TV for the day, or the week, or having to do extra chores, etc. Then you can choose an appropriate consequence when you need to.
If you really want to teach him to have respect for you and for himself, have him help you come up with rules for yourself also (such as no yelling, take out the trash daily, or whatever). Your consequences could be to put a quarter in a jar for yelling, or an extra chore like washing the kitchen windows, etc. Then, when you catch yourself (or he catches you) not following the rules, point it out to him & own up to it. Tell him you forgot, and broke a rule, and now you are going to wash the windows to help you remember next time. But you have to be consistent in following the rules & consequences yourself so he can see that it applies to everyone.
By coming up with a list of rules & consequences together, he will take more ownership in working together as a family and he will see that EVERYONE in the household has to work together. The most important thing you can do to get a child to respect you, is to respect THEM. Best of luck to you and your family.