D.T.
It's hard not to feel attached to the little ones when they come to you. Perhaps you connect with the kid based on the circumstances of his removal or just because you've had so much contact with him that you feel like he's your own.
It's not unexpected that you would feel fear too. Afterall you don't know what the child has been exposed to and what habits he might bring to the house. You have your own children to think about you know? No matter how fast and how good the counseling is you can't always keep the circle of damage from widening.
I would be concerned about my own kids getting attached too. at 4 and 5 they will be concerned about the new kid and when the child moves on they will miss him. It's hard to keep contact with foster children depending on the type of family that he comes from.
I commend you on your ability to take in other children. I wish I hadn't screwed up my background and I could help out like you are.
Prayers and positive thoughts coming your way.
D.