microwave handle repair ?

IMy handle was loose at the bottom of the handle. I am attempting to replace it. I opened the door and there is only a screw at the top, which I unscrewed. The handle then came completely away from the door. There is still a piece of broken plastic inside the bottom hole. It just spins around but will not come out. Is there something I should know regarding removing it, or do I just need to get a pair of pliers and force it out?? Thank you.