Make sure your replacement speaker has two rubber grommets installed on its left edge.

If it doesn't, remove the grommets from your old speaker and install them in your replacement.

Installing these can be tricky. Use your fingers to pinch the grommet and slide it into its notch.