It is displaying the symptoms of the video card going bad. Due to lead free solder and inadequate cooling in these imacs, the BGA array of solder balls connecting the video card (7300GT or 7600GT) to the logicboard fail due to whiskers and cracks. You'll need to replace it, best prices are on ebay at around $150-175. Try to get a 7600gt as these are much less prone to failure and if not get a more recent rom version of the 7300gt card.
Installation will requite torx screw drivers, and reapplying thermal paste.
In the mean time, dl smcfancontrol and raise the CPU and HDD fan speeds to help the card.
Booting into safe mode will also help.