D.B.
Your brother-in-law may be a pedophile, or he may be totally lacking in social skills so that he is drawn to children who cannot speak and engage him in conversation, which you say is a problem for him.
Holding kids is one thing, but putting fingers through openings in their clothing to rub them is something else again. Coming into your bedroom is poor form. I don't know if he doesn't understand boundaries or if he's a creep. But I would err on the side of caution, and I would trust your gut. I wouldn't allow this, and you (and your husband) shouldn't either. Taking them out of the room or behind the house is a deal breaker for me and a huge red flag.
I think it's ridiculous when people say, "Oh, he's not like that." We hear those comments every time a pedophile is arrested - no one can believe it, as if all child molesters look and act a certain way, when in fact the gentle and affectionate and funny ones are the ones who get closest to kids.
The main thing is, you have a husband problem. You only posted one other question some years ago, but it dealt with your husband's lack of sensitivity to your feelings. And here with the current question, you have the same problem. I actually agree with your husband about one thing, which is that you should see a mental health professional, preferably someone skilled in family counseling. If you're overreacting, you'll get perspective on that. But if you're not and if you have a potential predator in the family, you need a neutral and experienced expert to help you figure out a plan. Running away to your family for a few nights is understandable, but it's not going to work long term and it's not going to work when your kids are in school. I agree though, that you can't have one rule for some of the uncles and another rule for this one guy - that's just an unworkable situation. And your kids are going to start to pick up on this and see that you are nervous in these situations - and without your husband as a partner here, you're going to have a huge burden to carry.
If your in-laws have keys to your place and just come over any old time, with or without the brother-in-law, I can see that blowing up too. I wouldn't stand for that. But again, that's a marriage communication problem, and I think you have to start there. Meantime, cut back the family gatherings - it does sound like there are a whole lot and it's more than you can manage.