Kids live in the moment, especially three year olds.
I'd warn him 15 minutes ahead of time. "Son, we are going to pick up our things and leave soon, so get ready to stop in 15 minutes." He may not know what 15 minutes is, but he will learn after repetition.
That's one reason why I think potty training little kids too early is hard. They are so engrossed with what they are doing that they forget about the bathroom thing until they have to really go and then they can't make it on time.
Also, threatening and taking things away: power struggle you can't win. Three year old's are smart and kids learn very early how to push Mom's buttons. He knows you can't pick him up with the baby in the carrier so he's got you over a barrel.
Can you ask someone to hold the baby for you while you deal with him? Tantrums are a sign he's feeling out of control and while it's embarrassing, all parents go through it, whether it's asking for something at the grocery store and not getting it, or leaving play time with friends, etc.
I read somewhere, a long time ago, that you have to give toddlers a choice. "Do you want to wear the black shirt or the white shirt?" Not, "you are wearing the white shirt today." Because then for sure he will want to wear the black one, or even worse, the turtleneck on a hot day or plaid with stripes, etc.
My daughter got her little girl into gymnastics, because she was bebopping all over the house, not listening, and in general, wearing my daughter out every day. She wouldn't go to bed at night and was up till midnight tormenting her mom and dad with her demands to stay up and play. Since taking gymnastics, she's got a sense of pride in herself, a physical activity to do several times a week, and can follow directions from a teacher and has learned social interaction with other kids. She was in preschool for a while but think she's off for the summer.
Also, when he does something right, catch him at it and praise him! Try to ignore the tantrums and look bored. Say, "okay, son, see you later! Baby and I are going home for a snack!" Or just stand there and yawn. You might see him stop and look to see if you're watching him. If he sees you getting distressed and trying to calm him down, he will keep at it! Just take a "yeah, whatever" attitude and let him learn to calm himself down, as long as he's not hurting himself or others. Or if you can't stand it, ask someone to hold the baby or watch her in her car seat and cheerfully pick him and say, "time to go! We'll be back another day!" But IGNORE the tantrum.
Also, if it comes to food or clothing choices, let him have his way when it's possible. Be consistent with things like routines though, because that makes kids feel secure, even if they do have a tantrum. I know it's hard when you are stressed and embarrassed but he knows you are stressed and embarrassed! Kids aren't mean or doing it on purpose, it's just a way for them to break away from the baby stage and sometimes it's painful for Mom!
Do you have any routines, like at bedtime, where it's just you and him? I did a bath, with songs and talking, then drying off with more songs, then getting the jammies on (always an adventure, because my son knew jammies meant bedtime was coming soon so he'd try to run away), letting everyone smell his hair, maybe a relaxing TV show, or straight to the bedroom for story time and more talking. "Just stay 5 more minutes, Mom!" and this at 9 p.m. Yikes! Then it was hugs and lights out, music on the tape player, and maybe going back again to flip it over at 9:30.
So if you can get Dad to help with a routine, or have Dad help with the baby so you can spend even 20 minutes quiet time with him before bed, he might really enjoy that. Maybe you do that already. If you have to bring the baby in the room and nurse or feed, go ahead but make sure he gets his time.
Or you can buy a Wii or Dance Dance Revolution Mat. My daughter has the Wii fit as well as the gym class. Her daughter is 4 now but same stuff when she was 3. Good luck!!! Hang in there! It will pass eventually, just try and remain calm and have bananas on hand (they are supposed to relax you so if he won't eat one, you eat it!).