Here is my 2 cents, for what it is worth. I have been in a similar situation with my own sister and brother in Love((he wasn't always such!)), and she would go back and forth between my Husband and I being able to have the kids and not, because we were so involved in the welfare.
Now, I preface this by MY OWN PERSONAL situation. We called DFACS. Going to my sister, at THAT TIME, was to no avail. We knew the kids were in harms way, and we had to act. I never wanted my precious neice to look me in the eye and say...'You KNEW they were hurting me, and you LET THEM!'. So, we didn't. The result was removal from the home a couple of times, and in the end, well at least now anyway, the entire family has come to faith in Christ and is striving to live according to God's law and they are all doing much better.
If you can talk to your brother in a non threatening way, I advise that first, however, for me, that wouldn't have worked, because my sister was going to side with her Husband, reguardless, kwim?
Prayers are with you.
Heaven Bound,
A.
Mom2allofthem