After nursing, ice packs work like brakes do in a car. Although heat will help with milk let down so you can express it, after nursing, heat is like a gas pedal. Make sure you apply ice packs after each feeding until you reach the milk production level you want.
To kind of better understand what is causing all the massive milk production, amniotic fluid and breast milk are essentially the same thing. When women are pregnant our bodies super produce amniotic fluid month after month and the switch over to breast feeding seems to hit women like a fire hose. Essentially your body is not only stopping a river of amniotic fluid production, but it is also adjusting to the appetite of your new baby.
New babies need to nurse at night to help our body adjust. In fact, new babies do not start to experience REM sleep until about 6 weeks old. This is also the time when the uterus has returned to its pre-pregnant size.
You might also find this interesting. While women in America are single handedly taking on this frustrating and overwhelming challenge of transitioning at home alone, other cultures have it be standard procedure to have a support network present directly in the home as mothers bridge the divide between birth, a well recovered body, and a steady, well-paced milk supply. In Japan, for example, women stay in the hospital for an entire week being catered to during this process, they are fed foods of the highest nutritional quality, they are given back rubs and facials and then are assisted by their mothers at home for no less than a month. The government even has programs to assist women with no nearby family for up to two months following birth and even longer if necessary and this is done for up to 4 hours a day. I currently live here, gave birth to our fifth child here, and that is how I know this. While U.S. culture encourages mothers to prove themselves in a sort of maniac, false way by flying solo, the culture here (in regards to post partum recovery) is one of honoring and support. Every culture has its weak areas and its strong areas, of course.