You may not like the way it looks, but you could try just putting the crib mattress on the floor, and not even bothering with the crib. It may be easier to put him down gradually without waking him.
That's what I do for my baby for naps, and I can also lie next to him and nurse him for a little while until he falls asleep and then I'd sneak away. When he gets a little older, he'll be able to get up on his own, but that's better than climbing out of the crib and breaking an arm or leg. (I recently saw a 13 month old that broke her arm that way.) Just make sure the room is baby-safe and put a monitor on so you can hear him when he wakes up.
Oh, and make sure there's a rug or carpet underneath, so if he rolls off the mattress, he won't hit cold floor. (Mine has rolled off and not even woken up!)
Good luck.
P.S. One more tip, since you sound like you like to hold him to put him to sleep. My babe liked to fall asleep the same way, but as he got older, it got a little harder each day. It helped me to take him into a dark room and hold a running hairdryer in one hand. The white noise (sometimes) knocked him out within a few minutes and he got into a deeper sleep, so I could put him down easier.