If you were so unsure as to how things stand between you and your husband, then this was something you should have addressed BEFORE having a baby. Having a baby is not going to make your relationship rainbows and roses -- the opposite, it will strain your relationship if it's already in trouble, and it sounds like it is headed in that direction, due to your insecurity. If you thought he didn't care about babies, then you should have practiced some form of birth control and discussed what would happen if you did end up pregnant.
This is not the time for jealousy. When you marry or date someone with kids from another person, then you should know this person has a past, it is in the past, and there is no reason to bring that person into your relationship or compare yourself or your relationship to them. That doesn't mean he doesn't care about you or the baby, he may well be aware of the appointments and the fact he doesn't need to sit in the room for 2 hours for a routine screening. The fact he takes you to these appointments is commendable, many men don't go to a single appointment, or only for the ultrasound appointments. The fact he doesn't want you driving alone should prove he cares about you and your baby and your safety is his concern.
At the same time, if he has a job, he cannot possibly go to every single appointment so yes, I do think you're asking for too much. If he is your only source of income and he jeopardizes his job by taking off constantly for every appointment you have, he could get fired and then you lose your sole income and health insurance. Is it really necessary for him to go to every appointment and risk his job? It doesn't sound like it is necessary, or the norm, judging by the comments below. The reason he sits outside may be because he is able to work from the truck (make/take phone calls, respond to emails), something he would be unable to do while in the middle of a doctor's appointment, as you know. You don't mention the type of job he has, but this may be why he waits for you in the truck, not because he doesn't want to be around you.
You should make more of an effort to communicate and work on your relationship, that seems to be more crucial now than whether he goes to every doctor's appointment. You need to get to the bottom of why you feel he must be around all the time, and why whenever you don't get your way, you bring up his ex and compare your relationship to hers -- despite the fact you have the wedding band that she never did. This is unhealthy and will only cause him to become angry and resentful. You need to try to bring each other together, not create further division, and it is silly to drive a wedge into your relationship for the sole fact he isn't going to every doctor's appointment. Think of all the military wives, widows and single moms who go to appointments alone and turn out just fine. There is no reason why you can't go to your appointments alone like they have done throughout the decades, unless you're having a risky pregnancy or health issues that impede driving yourself and walking into a medical office unassisted.