The iPod touches (and most recent iPods, dating back to the 6th gen original iPod) have four contacts to allow stereo audio as well as the microphone and inline controls. (This uses a TRRS [tip-ring-ring-sleeve] plug instead of a TRS [tip-ring-sleeve] plug as usually seen on stereo headphones. More info at Wikipedia.)
Your headphones, since you said there were three connectors, are almost certainly TRS. If you can shine a flashlight into the headphone hole, there might be a bent pin in there that is causing a poor connection or a connection to the wrong segment. The only way I can think of to fix this, however, is to try to shove it around with a paperclip or other stiff wire.