Use the pointed end of a spudger to pry the volume control button assembly away from the edge of the case.

Do not remove the volume control button assembly from the device entirely; it is still connected to the logic board assembly by a soldered cable.