I have three and half year old twin boys and some nights a month, the scene is same as in your house. Other nights, I get them to tire out in the evening by riding bikes in the garage. I also give them warm milk just before bed time. They also take 30 minutes warm bath before bed time. I also tell them that when they eat or sleep, they grow, so they can get bigger clothes and big boy car seats and go to big boy school one day. I have encouraged healthy competition: verbal recognition to the boy with good behavior, and completely ignoring the boy with unwanted behaviour.
What I haven't tried, but may, esp if I were in your situation with a new baby .... I would tell them a fairy will put a sticker/coins under their pillow, for every night they went to bed with less than 3 or 4 minutes of talking in the bed (but no jumping). At the end of the week, count the stickers/coins and paste them in the kitchen, on their individual poster charts, for visible repetitive recognition for good behavior.
At three my twins were more difficult to handle before bed time, than they are at three and a half. I think it will get easier every few weeks.
Enjoy your new baby ..... first few years are an age of pure innocence ... its a blessing to get an opportunity to enjoy this age of your children!