I think the first thing you should do is throw out ALL pull ups. He is old enough, with a few restrictions, to not have to wear diapers. When I potty trained my boy, he was 3 months shy of being 3, and within 3 weeks of being potty trained (which took about 4 days), he was out of all diapers, even at night and bedtime. The trick is to cut off all liquids at a certain time. For my son, his bedtime was 8, so at 7 I would tell him to take his last drink of the night. If he asked for a drink later, I would say "no, I'm sorry, you already had your last drink of the night" and he was fine, it's not like a child will die of thirst when they just had a drink less than an hour beforehand. I took him potty right before naptime/bedtime, and waited until he went, which after a few times, they can pretty much go right away, and then also right when he first woke up we went straight to the bathroom.
I think what's happening is you're confusing him. Sure, it's easy for us to think that a child knows the difference between underwear and diapers, but a child's mind is not going to know when to let the urge go, and when it needs to hold back. You have to force him to ALWAYS have to get up and go to the bathroom when he has to poop or potty.
I wanted to comment on the lady above's response (a few above mine). She was saying how boys don't care to please people like girls do and was giving reasons why you should be more relaxed with your son. I just wanted to throw in that I am proof that you can't bundle all kids into certain stereotypes. What she said may be true for her kids, but to give you advice based on what she thinks all boys are like is just, well, a little ignorant (meant as actual definition, not in a degrading way). My son trained in about 4 days, my daughter took 7. My son loves to please me, my daughter loves to argue and test me with everything. My son cares about everything he does, and puts great effort into making coloring pages and drawings as perfect as he can, my daughter likes to make everything into a race. So, don't excuse his behavior as "just being a boy" because you'd only be fooling yourself.