Disrupted Sleep Schedule

This is going to be an unconventional answer, but hear me out. If it were me, I'd find a way to make sleeping outside of his room (in the hallway, where you found him) a secure option. As I said, unconventional, but let me explain:

You have a boundary with your son in regard to cosleeping, which is reasonable. Your son is in a tough situation: he wants something that he can't have and is too young to understand the "why's" of it (this has more to do with his brain development at this age than temperament, per se). Having a choice to sleep in his room or the hallway is better because it IS a choice, and usually people of all ages do better with disappointment when they have a choice as to how they will go forward. I have also seen this play out in several families I nannied with, so I will give you a few choices:

First, no matter what you choose to do, keep your bedtime/naptime routines consistent. Then, give him two clear places he might rest: in the hallway with a 'cozy nest', which is usually a couple blankets and pillow on something soft--sleepingbag, foam pad-- anything conducive to rest OR he may sleep in his room. If it were me, I would put a gate at the top of the stairs anyway, because youngsters and stairs aren't a great combination in general. I'd read his stories and songs and snuggle him down to bed in his chosen spot, and then tell him it's time to sleep. He can sleep in the hallway or in his room. When he bounces out of bed, simply take him back to bed and tell him "It's time for sleeping now. We'll play in the morning." I know this is will be a test of your personal fortitude, but keep taking him back to his bed with little to no comment. He isn't at an age where reasons will matter to him, so your explanations actually only result in giving extra attention, which is what he's wanting. Punishment is likely not going to produce anything positive in this matter, because he's wanting the connection and attention, and we don't want to punish our children for desiring closeness with us.:) So I'd skip the time-outs, spankings, or other activities that can really become out-of-proportion distractions from what you want him to do, which is ultimately, to sleep on his own.

I would certainly caution any parent against locking a child into a room. This is a terrifying experience for a child and is also a fire hazard--if your child needs to come and get you because they are ill or in an emergency situation, they need to be able to do so. If you think this is a bit soft, let me ask this: would You as an adult like to be locked into a room? What if you were ill or perceived a danger? Imagine truly being in this situation, and let yourself experience those feelings. How would you respond? Children have very immature, developing brains, and they experience access to their parents/caregivers as intrinsic to their very survival. This is more to do with their biology than their being willful at times, and it's good to remember that they often experience separation/loss as very real pain in their young brains. This could be a very real reason for his pounding and banging on the door and screaming, so make sure he always has a way to get out of his room.

Some families I've worked with have let their children build cozy nests in the parent's bedroom on the floor; others, in the hallway. Some parents go in and lay down for a while with their children to help them relax. (This is useful if the child perceives they are missing out on something fun.) I've often seen that children (especially older ones) do well with a stack of books to look at when they can't rest. This way, they aren't being forced to sleep when their body isn't ready, but they also have a positive "something to do" that is usually suitable for the time of day. (Naptime--quietly looking at books isn't disruptive to other children, and this is a good practice for "quiet time", when children are older and drop their naps but we parents need a break.)

The last suggestion I'd make is to find as much time as you can for positive connection with your son during the daytime. I don't know what your end-of-day routine looks like, but it's been my observation that our bedtime routines can be more about the "doing" of getting ready for bed (toothbrushing, bathtime, etc.) than a relaxing time to snuggle and connect. Two years old is still toddler time, and toddlers really need us a lot. Even more so than some stages of development, because while they are practicing separation, they do try to balance this by clinging to us even more. It feels extreme to us on some days--and it likely it-- and remembering that This Too Really Truly Will Pass is helpful. Give as much positive attention as you can during the day, as much patient guidance as you and your spouse can muster, and give it time--it will eventually soften and resolve. It might take a while, but I've seen it happen again and again. Good luck!

i realize this is probably not what you want to hear, but i would strongly advise letting him sleep with you, at least for now. children need their parents both day and night, and sometimes need them much more at night. it sounds like he is terrified, for a reason you may not understand but nevertheless very real and important. you say that you are "not co-sleepers" with your kids unless they are ill - well he may not be physically ill but he is clearly hurting very much and he really needs you. he will not be in your bed forever, but he will be a much more secure child and adult if you meet his nighttime needs now.

You're doing everything you can. He's just letting you know that he's not happy. Tough it out and he'll get tired of it all, but only if you don't lose your temper or give in. If you do, next time he doesn't get what he wants when he wants it, the tantrums will be worse. Good luck, and buy some earplugs!