Brooke's suggestions are excellent given your son's age and that he is mostly potty trained already. Her pediatrician has the right idea - it's something to work on every day but it is a gradual, no-stress, logical transition. Also, others who have said to make sure there's no pressure/negativity surrounding pooping in his diapers are absolutely right too - your approach must remain relaxed, matter-of-fact, loving, patient, and supportive. And consistent - that people poop in the bathroom, that poop is flushed down the toilet (even if it goes first into the diaper). I definitely do NOT think it would be a good idea to just stop any efforts to potty train for poop at this point - the vast majority of kids do not just train themselves. It's up to us as parents to support them through learning this skill. Have you talked to your 4 year old about how it's more comfortable and healthier to poop regularly instead of holding it? Have you guys read that book called "Everyone Poops" by Taro Gomi?
http://www.kanemiller.com/book.asp?sku=25&sc=1
As for being scared, perhaps he is not scared of monsters in the toilet - maybe he is scared of feeling his poop come out? If he's ever had uncomfortable bowel movements/constipation, that could make him apprehensive about going too. Lastly, it's really important to:
* observe & learn his body language prior to when he needs to poop (so you can recognize when he needs to go and quietly, non-chalantly help him get to the toilet - you maybe be able to predict this better than he can, and then help him recognize this behavior in himself)
* try to see what position he's in when he actually poops - does he remain standing? is he in a deep squat? - and then provide him with a toilet + step stool or small potty that would allow him to be in that same position when he does eventually try without the diaper (or with a hole cut out of the diaper)
* I know it might sound a little awkward, but encourage him to sit on the toilet whenever YOU have to go - do it together, read a book together while you're there to distract him enough so that he can relax and perhaps poop, and if you have to poop, talk him through what it's like (so that he's less scared, so that he knows it's ok)
Just stay positive and know that you and your son are absolutely able to work together as a team to help him learn to poop in the toilet and not in his diaper. As you said, he's quite capable (writing his name, adding on his fingers, toilet trained for pee), you know he can do it, and as long as you remain no-big-deal, supportive, loving, gradual and committed to the process, he will be toilet independent sooner than you think!