I discovered this not so long ago, when I started using Skype. I get notification sounds even though my speakers are on mute. The same thing happens with email. Suddenly there's a notification/alert sound for everything during mute. Is there a setting where I can disable this "feature"?
The most likely explanation for this is that you have multiple sound output devices. When you mute, only the current output devices is muted — the rest are not.
The next time you hear a sound when you think you shouldn't, go to Sound Preferences > Output and check your Output devices one by one. It's almost certain that at least one of them isn't muted (a headset, for example), and that's how you're hearing the mystery sound.