My advice to you would be to set a routine and stick to it! Consistency is the key. Always (or almost always), eat dinner close to the same time, have a warm bath, put on pajamas, read a few (2 at most) books, and then night night time. If you stick to this routine, things are predictable for your son and he knows what is expected of him within that routine. It takes some time for him to realize that this is how things are going to be, but once it's in place, it should work to your advantage (and his, really). You may already have a routine in place, but it's helpful to hear about it again.
If and when your son gets out of bed, I would very quietly (without a lot of social interaction), go in there and place him in bed once again, and then walk out. Having you do this over and over means that "play time" is over, and shows that your priority is to get him into bed versus giving into his attention getting behavior and reading to him more, or getting him a drink or whatever. (Kind of like the Super Nanny does)
I have read on this site that many moms/dads put up a gate in the child's bedroom door opening so that they can not exit their room...that may help curtail his coming downstairs too. Because the door is open, he won't feel "trapped." Could be helpful.
My only other idea is maybe to remove any toys or things that he'd want to get out of bed to play with (other than books). Perhaps giving him the message that his room is a sleeping place at that time of day could keep him from wanting to play so much. If there is nothing for him to do when he gets out of bed (except maybe look at books), he may be more inclined to stay in bed.
If he's not scared, what would happen if you didn't go into his room? How long would he stay awake in there before wearing himself out? Would he finally get so tired that he'd fall asleep on his own? He may be prolonging bedtime to spend time with you and your hubby...
Good luck - sweet dreams and good sleep to you all