Yes, that is bad.
Apple's fan controls favor quiet over safety and tend to key off the CPU temperature, ignoring the fact that the GPU can go far out of spec even while the CPU remains relatively cool. 3D games in particular can burn out the GPU because of the native control's bias toward CPU temperature.
If you are going to play video games on an iMac or MacBook, I strongly recommend obtaining fan control software and setting a steep profile so that the fans spin aggressively with CPU temperature, thereby keeping the GPU from going out of spec. If you are playing games in Boot Camp, having a separate fan controller is absolutely essential.
It sounds like you may already have some damage to the GPU or motherboard. If you are still under warranty or AppleCare, try to find an artifact that you can reproduce at the Genius bar and get the GPU/motherboard replaced. Remember that the system will be cold when you get there, so it may take a while to reproduce heat problems. The Geniuses like to use the iTunes visualizer as a test case, but if you have an offline game that produces problems that will work as well.
In any case, increasing the fan speed to keep the GPU temperature down while playing 3D games will greatly extend the life of your system.