Is it possible to delete a song from a playlist with a 'swipe to delete'? I thought this was possible, but I am not able to do this. I'm running iOS 5.0.1.
You're right. You can't delete songs in a playlist with a swipe. Only the way Nathan described. Hit the "Edit" button on top of the window, and remove the songs with the familiar red road sign.
You should be able to do this with a swipe on a whole Playlists, Artists, Songs, Albums, …
And make note you can't delete songs with a swipe from iCloud.