I do not have a schematic handy, but if it is one of the large(ish) electrolytic capacitors, and its top has bulged or split, replace it.
These capacitors are part of the circuits that supply power to the processor unit(s), and when they go bad they can indeed cause the processor(s) to "hang" or overheat. My personal experience is that in about half of cases where computers are acting up, the cause is bad power supply capacitors.
It would be a good idea to check the other capacitors too. When one goes, the others usually are not far behind. Keep in mind that a capacitor may be bad even if it has not (yet) bulged or leaked. The only way to be sure if a capacitor has gone bad is to test its ESR (Equivalent Series Resistance). While standard multimeters cannot measure this parameter, you can [http://ludens.cl/Electron/esr/esr.html|build a simple but effective ESR meter yourself] for a few bucks. I highly recommend it, this little instrument will save you a lot of time and trouble if you regularly work on any type of switching power supply circuit.
Caveat: replacing electrolytic capacitors on a motherboard is not an easy task. Getting them out without damaging the motherboard being the major problem.