Anybody has an idea why videos taken with an iPod Touch (or iPhone 4S) play upside down on my Windows PC with Windows Media Player? However with QuickTime, videos are oriented correctly.
Windows Media Player is either ignoring or misinterpreting the orientation information in the video's metadata.
Since QuickTime better supports the format that your iPhone records in, it makes sense that it's doing a better job.
If you want to use Windows Media Player, your best bet is probably to convert the video into another format that WMP supports better. One app that allows you to do this is HandBrake.
Otherwise, you should be fine using QuickTime, iTunes, or a Mac.