What is so wrong with it if you don't even notice they are there? I bet once you stop making such a big deal about how they should not be in there, they will stop coming in!
Don't make your room off limits, it only makes them scared that you are shutting them out. They can't verbalize this, but they feel like you are withholding love, even if that is not your intention. They are just trying to hold onto it!
Make up a space for them to sleep near you that is acceptable like the pp said, sleeping bags at the foot of your bed or next to it. If you let them know you care and they are welcome, it won't be such a big deal. You know how kids don't want something until you say they CAN'T have it!
You could also try to see if there is anything you can do to make their rooms more welcome, a special night light, glow in the dark stars on the ceiling.
Try letting them camp out in front of the Christmas tree on Christmas Eve! Set up sleeping bags and make it special. We always did that when we were little!
If it really bothers you during the week, how about letting them sleep with you on the weekends and holidays? Maybe they would look forward to Friday night when you all go to bed together. It might help them stay in their rooms on school nights. Just make sure you don't lock your door again! That is really scary to a kid, to not have access to a parent! I hope you can find a happy solution. They are still young and they won't be in your bed forever! I promise!