This sounds ridiculously simple, but it has worked for me in the past. Try checking and then unchecking Universal Access » Hearing » Flash the screen when an alert sound occurs -- this really sounds like that is the behavior in question, and sometimes setting it and the unsetting it resets to the desired behavior. (Note, this might be under Audio in newer versions of Mac OS.)
This basically checks that the preference file can be written to have a no then a yes and will clear up all sorts of minor issues in how apps can store preferences. The next step is to delete (or move to the desktop) the preference file for the affected application. In this case, the system stores this flash preference here:
~/Library/Preferences/com.apple.universalaccess.plist
At this point, you might need to expand to see if rebooting, trying a new user account helps isolate the issue.
