For example, I would like for iTunes to play thru my iMac's speakers, but iChat's notifications thru my headphones.
The easiest way to do this depends on the application supporting it;
Some applications have a sound output (and input) picker that allows you to choose the output device, like the official Last.fm app.
(My apologies for the poor example considering I only have one output source.)
If an application is written to support it, you can simply change the device as shown above. Otherwise you'll have to rely on premium (paid) apps, such as one of Ambrosia SW's "Wiretap" series of apps. Though those are pricy, $69 for WireTap Studio and $129 for WireTap Anywhere.