If you were to remove the battery from an iPod touch, and then connect it to a computer, would you still be able to play music on it?
Probably not - In general, iPod & iPhone devices won't be usable when connected to a computer because they are trying to sync with the computer.
Instead, try getting an external speaker dock of some sort and drop the iPod into that. It's designed to power the iPod while playing.
I'm certain that this will work in the case of a dead battery. In the case of the battery actually being removed, there's a small chance that the iPod's on-board electronics will not work correctly, even if externally powered. This isn't likely, but it's possible that without a battery, the charging circuit will cause something else to go wrong. Again, not likely, but not impossible.
If you have a device with a dead battery, and you don't want to replace the battery, just leave it in there, drop it in a dock and play on.