Like everyone here has already said, massage is definitely ok during pregnancy--in fact, if you'd never done massage before as a method of stress relief and headache control, THIS would be the time I would recommend starting it. I am trained as a midwife (not practicing due to raising my own kids and needing a more consistently paying job), and my partner is an LMT. Just be sure you do tell your massage therapist that you are pregnant because there are a few acupressure points they will want to avoid, as well as ways to massage that specifically help pregnant women in different trimesters of pregnancy (no one mentioned THAT yet! ;)
I would also suggest a few other methods of headache control. I suffer from migraines occasionally, and pregnancy was no exception. Having a massage therapist in the bedroom at 3 am was a wonderful thing, no question, but even he couldn't always solve it and I didn't want to take medications during pregnancy. So I learned about herbs--in fact, that was what originally led me to midwifery.
Here are some that are safe during pregnancy and work in various ways to reduce/cure/prevent headaches:
Passionflower--my A number One herb of all time! You can buy this in tincture form (don't worry about the tiny amount of alcohol involved, unless you already have issues with alcohol--no it will NOT harm the baby, and the alcohol not only draws out the herbal constituents the best, it also causes your body to absorb and utilize them immediately if you place a dropperful under the tongue, which has receptor sites that don't require stomach digestion. If you do have alcohol issues, you can also get this in a glycerin base, but it's not quite as effective). Passionflower is not only a beautiful plant you can grow in temperate regions, but it is a natural vasodilator (relaxes your blood vessels), releasing tension in the neck and shoulders, a primary point where the nervous system gets "tight" leading to headaches. There is NO danger in pregnancy, and nothing habit-forming, so you can take this as needed. I survived with this stuff!
Valerian root--also best in tincture form. This is the basis plant for valium, though when taken in plant form, with all it's natural constituents, it does not have the intensity or the habit forming nature of the drug. Also not dangerous in pregnancy, but you shouldn't do more than a couple of dropperfuls per day. This is mildly sedative as well as working for those "front of the brain" headaches. Good for night time, but not bad for day if you need it. If you grow it, the plant forms tall stalks with umbrella like white flowers, and you will notice that just as the sun is setting, it will send a soft, soothing sent all through your yard--and that is it's action internally!
Skullcap--Again, tincture. Again, probably 2-3 times per day, no more than a few days in a row. Excellent when nothing else works. Can be mildly habit-forming, but nothing like pharmaceuticals.
Motherwort--every mother's friend, literally "Mother Herb". Can be taken at will, and is good for mild stress/tension/hormonal mood swings. Continue this one postpartum!
Good luck!