Family or not you do not know them. You do not know if they are safe or not. If it were me I would not have my children around them for a long time. You must get to know them enough to evaluate what type of people they are. There are reasons you do not talk to your father, you don't know what character traits they have recieved from him. If your brothers want to meet you, than thats fine you and your husband should. But not your kids yet. You brothers should understand that the siblings, you and your brothers, need a relationship first. Then the kids can build one later.
Google their names, go on the states website they live in and see if the have a state and federal prision list. I was in California and was learning about my step father in law. I called an 800 number for state and a different 800 number for federal to see if he were incarcerated at any time. They just needed his name and birthdate. Learn as much as you can first and decide later.
What I did with my son. He didn't understand our family. Some do not speak to eachother, some are remarried, my dad is cut off from me. I got legos. My biological parents were a red (dad) and blue (mom) legos squished together, i used two different shapes for different people. Then made me and my two sisters different shapes but blue red mix of legos. I showed him they divorced and my mom remarried. My step dad was yellow. They had my brother he was blue and yellow different shape though. He saw that me and my sisters all had blue as did my brother, but we were a little different. I took myself (red blue) and showed I married his father (my husband was two colors from his parents like green gray). When i showed we had him and his sister they were red blue and green grey mixes in two shapes. I did the same with my husbands side of the family. Ever since then he has understood. He doesn't understand divorce completely, I just explained they had problems and the only way to fix them was to be away from eachother. I did not blame anyone for it, just left it as problems. I did not want to villify anyone or place blame, he just needed to understand where everyone stood in the family including himself.
I hope this all made sense. I know my friend just drew and colored a family tree, but I have divorce and step parents to factor in. Mine would have ended up looking like a spider web !
I hope that helped!
J.