As you might have feared based upon the countless other disappointing Apple-related answers, the answer to your question is "You can't". You cannot play a file in iTunes without adding it to your library as well. Moreover, a converted version of your file is generated and copied into your iTunes library folder.
Seeing as how you have Alfred working for you, and are otherwise willing to consider workarounds, I would suggest these potential [partial] workarounds:
Associate each desired file type with a batch process to stream the desired audio through a third-party podcasting application to a specific address, and launch iTunes with the proper arguments to play the streaming audio from the appropriate address.
Associate each desired file type with a batch process that modifies the file before launching iTunes, so that, once another partial solution is found, all files launched in such a way are easier to target programmatically, eg:
-- prepend or append some distinctive string to the filename
-- for mp3s, modify some rarely-used ID3 tag value
Deleting the converted file that iTunes automatically creates when adding audio files to its library. (Obtaining the exact name with no margin for error may prove challenging however, nor will it remove the file's entry in any external library inventory file if there is one.)
Settle for programming some process to merely uncheck the box in iTunes next to the file, which will prevent it from being played unless you explicitly ask for it to be played. (This will not stop the file from being synced to your iPhone or iPod though, and I am not sure if its unchecked nature on your computer will prevent it from being shuffled in on your iPod.)
(Also keep in mind that Automator (included with OS X in the Utilities Application folder), particularly its "Watch Me Do" feature, may offer some key functionality in creating executable shortcuts to actions you take manually, which may in deed provide an important part of the solution to your endeavor.)