Disclaimer: I tried my best to include all information I could possibly give, but I could have mistakenly left something out. Please be careful, and I won't be responsible for anything that goes awry with your device. These steps worked for me, but sh** happens.
Apple isn't signing the iOS 5.0.1 .ipsw's anymore, so you're going to have to make sure you have your SHSH blobs for 5.0.1 by going into Cydia and seeing if 5.0.1 is listed up at the top:
If you see it there, you're good to go to the next step. The way Apple signs .ipsw's has changed with iOS 5, so if you've previously done a downgrade/unsigned restore, you'll notice many steps that aren't the same as the ones you're used to. You might run into a problem if your SHSH blobs weren't saved properly. It's not your fault, and if you do encounter any problems, stop following this tutorial. You won't be able to fix it. Nobody can yet; that's the sad truth. I do, however, suggest that you save your SHSH blobs using iFaith. That way, if anything goes terribly wrong, you'll still be able to restore to any version of iOS that you have the SHSH blob for.
You'll also need iFaith for another step, so make sure you have it. Once you have everything saved and the programs downloaded, you need to download the iOS 5.0.1 .ipsw file here. You're going to want to make sure you get the correct version for your device. Once that's finished downloading, plug your device into your computer, open iFaith, and dump your SHSH blobs (if you haven't already done so). Once they're saved, you'll be taken back to the main menu in iFaith. Create a signed .ipsw with your SHSH blobs by selecting the respective option and following the on-screen instructions. Once it's created, open up iTunes, navigate to your device from the menu on the left, and you'll see the Upgrade and Restore buttons. You need to hold down the Option key (Shift on Windows) and click Restore.
You'll then need to select the .ipsw file you created earlier in the window that comes up. Once selected, it'll start restoring your device back to your selected version. Now you'll just need to wait for it to finish. It's easy to mess up, so please be careful.
You should now have a brand new (non-JailBroken) install of iOS 5.0.1. If you need it JailBroken again, just do it the same way you did before (with redsn0w).
