Any leftover residue will compromise your iPhone's seal. Be sure to clean the frame thoroughly.

Wrap a lint-free or microfiber cloth over the point of your spudger and apply a few drops of highly-concentrated isopropyl alcohol (over 90%) to the cloth.

Wipe in one direction along the perimeter of the frame to clean the adhesive residue.