T. - As others have already said, go ahead and use the self cleaning feature, turn on your hood fan if you have one as the oven heats up over 500 degrees to burn off the mess. Don't let your little one near the oven as the outside obviously gets really hot with inside temps being so high.
I would strongly suggest you do not use oven cleaner. It is one of the most caustic and dangerous cleaners you can have in your home, along with drain-o products. (It contains Ethers, ethylene glycol, lye (sodium and potassium hydroxide), methylene chloride, petroleum distillates among other terrible ingredients.)
If my self cleaning oven still has spots that need work (rarely does, by the way!) I use plain old baking soda. (In fact, I use that for most scrubbing jobs in my house along with white vinegar for cleaning and disinfecting.) Make a paste with some water and let it set for an hour. Yes, it takes more elbow grease than Easy Off, but your lungs will thank you for it!!