K.C.
It sounds like you've got a massive case of plugged ducts, and the only way I know from experience to relieve that is frequent pumping/nursing. Which defeats your purpose.
I'd watch for infection (redness, fever), and contact a lactation consultant or La Leche League leader. Things I've heard of for drying up milk are cabbage leaves in the bra and taking a decongestant like Sudafed for a few days (the original, over-the-counter stuff would be more effective).
If you're trying to get the milk out, you should use a warm compress or hand-express in a hot shower. If you're just trying to relieve pain a cold compress would be the thing, as well as your pain reliever of choice. But if you have a fever, head straight to the doctor!