She might be going through this even if there weren't a new baby, but the new baby makes her much more vulnerable to the restless-bedtime syndrome. Expecting her to get drinks of water herself because it's next to her bed isn't really going to work, as you've found -- it's you she wants, not truly the water itself.
Your husband needs to help out if he isn't already. One you takes the baby for bedtime, the other handles your daughter, so just one of you is not doing all the work every night. Be sure you both handle things the same way, though, or she will be confused from one night to the next.
I wouldn't treat it as reason for discipline; she's not trying to manipulate you or be "naughty," she's realized her world is rocked and she's looking for reassurance. Yes, you do have to work on a routine. Try putting her to bed, returning in five minutes, then 10 minutes after that, then 20 minutes after that last time. This way she knows you're there but knows she's not to come after you, you'll be checking in on her instead. Some kids stay wakeful through this for a while but do get to sleep once they learn that mom and dad really do return.
You could check in with the SuperNanny books by Jo Frost. She's really sensible about bedtime routines. But again, I'd not let this become a discipline issue -- that will be tough, since it is sure frustrating, but she does indeed need to "re-routine."