I had a lot of trouble with that. The trim pieces are held in with spring clips molded into the trim itself. The trim pieces fit into grooves in the bumper face, so they should be carefully removed from the inside of the bumper shell. IIRC the bumper cover just has a few push-pin plastic rivets to hold it onto the core piece. 
Here's the diagram at RealOEM. Mine only had three rivets (part 2) and was missing all the shim washers (part 3).
The trim pieces are still fairly difficult to get at from the inside of the bumper shell. Maybe a modified plastic trim tool or even flat-bladed screwdriver could serve as a drift? I pushed a few of them too hard and made little cracks in the bumper around the insertion slots.