With my son, the thing that really made an impression was being a "big boy." while I don't normally like to play this up, in the case of weaning at 3 I found it a great help because he really responded to wanting to be a big boy and I just explained that he was a big enough boy to not need mommy milk anymore. I started this a few months before I planned to wean him, and would remind him that "soon you'll be a big enough boy to not need mommy milk" when he would nurse and also was sure to mention that there would still be plenty of snuggling and time on mommy's lap even without nursing.
When the time came to wean, I'd offer a few drink choices instead of nursing when he asked. There were a few tears that were easily quieted by a drink and a hug or snuggle, and within 2 days and asking only 2-3 times, he never asked again. Naptime routines changed since nursing him down was no longer an option, but a snuggle in bed and telling him a few stories worked fine after a few days of adjustment.
The biggest mistake you could make is waffling, I think. If you give in if he cries or complains and nurse again, he's going to continue to cry and complain to get what he wants. Distraction and substitution (a sippy cup or glass with juice or milk and plenty of hugs and reassurance) was key in taking care of those few moments I felt like giving in.