Hi M,
Not sure how old your baby is, but be careful about too much heat and bundling up. There should be plenty of air circulation (use a fan) and the room should be on the cool side (in the 60s) at night. This will help prevent SIDS!
Also, ideally you should not use a bumper for the same reason - sometimes babies can smoosh their little faces against it or get their heads caught in the bumper and then can't breathe. That said, if you don't want to take it out right away, definitely take it out before your child can use it as a stepstool to get out of the crib.
Lastly - I recommend using a sleep sack instead of a blanket. They come in all sizes, from infant to toddler, and you can find them at Babies R us and online. The sleep sack zips up and doesn't fall off if your baby rolls around. I used one for my child and it worked great.
You should be able to put your child in regular jammies (long or short, depending on the season), then the sleep sack and be good to go. (They also make a fleece sleep sack for winter, but I found it's way too warm for Southern California winters.)