I just went through this about a month ago with my 3 year old. He would NOT let me near his 2 slivers for 2 days. Coincidentally we were at the doctor for my other son's checkup and they were more concerned with my 3 year old's slivers than anything because they were started to get red and a little infected. So please, don't leave it in...or at least monitor it closely.
I watched it for 2 days...he took long baths to try to soak it out, he actually stood and held is hand in Epson Salt water for like 45 minutes one night because he didn't want me touching it and that didn't work either. Nothing was working.
So finally I tried again to take it out. I just talked and talked and talked him through it. He would start to freak out and then I'd back off and we'd talk more about it. I let him feel like he was in control...I let him help hold the tweezers, I let him try it on me, etc. I did everything I had to...told him it would only hurt for a second like a shot but then it would feel MUCH better when they came out; I reminded him that we would have to go to the hospital if he didn't let me take them out; I bribed him with a reward if he let me take them out, etc. He was pretty good about it - it just took a lot of negotiating.
Good luck :)