Psp battery and console issue

My psp has been sitting away for years and when I found it in my closet I noticed the battery was very big and popped the back case open. I took it out and ordered a new battery. (not a original psp Sony battery )

Once the battery came in the mail I put the new battery in but it won't completely fit. The console itself doesn't stay on because of this issue I think.

I don't know if the console got heat damage. Need help.

Please and thank you.