does the battery have cells?

A lithium polymer battery can have "Cells" but at these compact sizes they are a single cell. However battery cells vary in capacity, so counting cells is not a good unit of measurement, instead you rate batteries by milliAmp-hours. or mAh. Which is the amount of amps the battery can put out over an hour of time.

A 3rd cen iPod touch has a 750 mAh battery.

Since the iPod doesn't pull 750mA to run, it can run for several hours on said battery. The actual amount of battery life depends the following factors:

• What app you are running, and how much CPU that app takes.
• How much that app accesses memory or other parts of the iPod such as the accelerometer.
• Whether or not you have Wifi or bluetooth on, and if your app is accessing those.
• How many apps you have running that use push notifications.
• The brightness of your screen
• The volume of your sound
• The age of the battery, temperature in the room, and how well the battery was taken care of.
• And more...

As Andrew said, replacing the battery with a fresh one is the only solution to getting more battery life, if your battery is indeed dying. You won't be able to find any legitimate expanded batteries that fit inside your iPod touch, apple already crammed in the largest battery they could fit, you won't find a higher capacity battery that is physically the same size.

After that the only other option is to use battery extenders that attached to the iPod touch.