It sounds like you're describing my life, when my son was born! We went through the exact same thing, and we did just what you're doing.
The key here is to listen to your baby. (Which you're doing perfectly!) I think some babies just need to be held constantly at first, and can't sleep well alone. For the first three months, my son would SCREAM for an hour in my arms before he fell asleep in the co-sleeper at night. And he would end up in bed with me after the first waking, then sleep well throughout the night. The rest of the time, he would nap just fine as long as I was holding him! I was pooped.
Around your son's age, my baby started sleeping less well in bed with us. And us, too! So I started using my sling to help him fall asleep, and then I'd place him in a pack-n-play right next to our bed. Sometimes he'd come back to sleep with us after a nursing, and sometimes not.
Over the next couple of weeks, we inched the pack-n-play towards the far wall of our bedroom. Once his "mini crib" was across the room, we were ready to help him transfer to his real crib. That was another long process, where he would start in the real crib and then sometimes end up in the pack-n-play.
He did not take to the crib until I let him soothe himself to sleep, however. But that's a whole different issue! :)
The key to this long, rambling answer is- keep listening to your son! By 4 months, my son was self-soothing and sleeping through the night (11-12 hours!). You can tell, when he's not sleeping well in bed with you anymore. And you can tell, when he's big enough to handle a little fussing and thus learn to self soothe.
And you can tell, when he's ready to sleep alone in his big boy crib. No need to rush it- he'll let you know when he's ready!
PS My son got way less fussy when I figured out, at 12 weeks, that I was putting him to bed too late. 8:00 became 6:00 and WOW what a difference!