Yep, this sounds about normal. I also sometimes use videos as a distraction for my kids, but most of the time feel guilty about it. He probably is just begging for attention from you, so try to give him as much as possible, but it's better (imo) to plop him in front of the TV for half an hour than for you to have a screaming melt-down because you can't do ten things at once. Just try not to make a habit of it, using it sparingly -- for two reasons: 1) if he gets used to watching TV, then he won't be a zombie when you want him to, and 2) too much TV isn't good for him and you need to remain active and involved in his life. Maybe you can find an alternate activity that he likes that you can use at such times. Because my kids started drawing on the walls, I threw away their crayons a year ago; now, I've got some more and let them color during church (to keep them quiet) and also sometimes at home. Since they don't do it much, they are entertained for a long period of time when they do color, and I get a break w/o resorting to videos. You can also pack away a box of his toys, and then when you get them out a month later, they'll be "new again" and he'll play with them for a while.
Also try to incorporate him in your activities -- with or without the baby. While you're nursing the baby, you can read to him or tell him a story, or watch him quietly play. He can "help" you with laundry -- depending on how your washer & dryer are set up, he can hand you clothes out of the dirty clothes basket, or put them into the washer or dryer if they're front-loading, or you can lift him up so he can put the clothes in a top-loading washer, and he can fold small simple articles like washcloths and dish towels, or pick out all of his underwear, socks, or other clothes. When you're cooking supper, he can stir, or pour measured ingredients into bowls, or you can give him "his own" bowls and stuff to pretend to cook, or he can pretend to wash dishes while you're cooking. Unless he's scared of the vacuum cleaner, he'll probably enjoy "helping" with that, too; and I know he'd love to sweep and mop for you. You can give him a dust-rag so he can dust the lower half of a curio cabinet or set of bookshelves while you get the upper half. There's a lot of ways to help him get involved, which will make him feel not quite so displaced by the baby, and will probably help his attitude.