I was also a student going through finals and pregnant at the same time. I wanted to quit for the same reasons - I didn't want my child to suffer withdrawals, plus, the unknown side-effects of asparatame and what I later learned - the Calcium and iron absorption-blocking properties of soda and tea.
Anyhow, I was able to reduce my caffeine intake during pregnancy by purchasing the smaller bottles of soda and drinking all-natural sodas and/or mixing my own with seltzer and 100% juice.
When I finally quit my diet soda and tea habit, however, it was by going cold turkey. The headaches only lasted a couple of days. I don't think I took any Tylenol, but I have since and my body is really responsive to it since I hadn't been taking any regularly during and after pregnancy - only 500mg. I wouldn't recommend taking Excedrin, because as you know, it contains caffeine and may not be approved during pregnancy.
Congratulations on getting to this point. Being caffeine-free is very liberating.