Push the PRAM battery in and pull it up. You'll have to push the battery in further than you'd expect in order to get it to pop free.

When inserting the new battery, make sure the side with writing along the edges faces the black plastic holder. You should also insure that the metal connectors make contact with the battery (you can bend them forward if they do not).