The beauty of tagging, as you may know, is that you can easily assign multiple categorizations and topics to each item, rather than just having them exist at one location which defines it as a type or part of a static collection. I still use the shallow hierarchy of folders that drill down to individual projects and topics, so it's not a "one pile" deal. I couldn't function like that; it's difficult to weed, and if metadata is lost, so is the file, essentially. I use folders to maintain filesystem sanity. I use tags and other metadata to maintain my sanity.
I also use tags on iCal events, emails, web sites and other elements of my system. They're easy to filter in searches and allow me to make a project "folder" that contains more types of data than Finder would normally make convenient.
[...]
The secret, which I've learned from a few years of doing this, is to tag intelligently, sparingly and consistently. Being liberal with tags like "important" or "flagged" just means more upkeep and reviews that have to be done to keep the tags relevant. While tags like "neat" or "supercool" seem silly to me, I use one "inspiration" tag on anything that I'm saving just because it's, well, inspirational. I have a Smart Folder in Finder that pulls up everything with that tag, and then I can filter the files based on date, type, etc. If I add additional tags, they are pretty general, such as "color," "webdesign," or "productivity." These are big buckets I can dive into when I need an idea, or a kick in the pants. The tags on project and research files tend to be more specific, and I can drill down to precise sets of files by typing a few words in Tags or HoudahSpot.
To keep the cruft down, I use tags which aren't already defined by other metadata like the creation or modified date, the filetype, the filename, etc. I do tag files with the name of their parent project or topic, even though it's often redundant to the folder hierarchy. This allows me to tie projects together across multiple folders, disks or even servers. Sometimes a file is more strongly tied to a topic than it is to an individual project. Say it's a general reference item or a stock photo that I gathered while working on a specific project, but that I know I'll need again. I put it into a reference folder or photo collection, and tag it with the related project. It's easy to view it in the context of the project with a tag search, but part of a larger collection of similar items at the same time.
When in doubt, I usually choose tags based on the first thought that pops into my head. Chances are, when I'm looking for it again in a year, I'll have the same thought. It doesn't always work, but I've had a pretty good success rate over the last couple of years. As often as possible I try to reuse tags; a tag that reveals one or two files is a useless tag. My "Yearly Review" includes weeding out tags that have very few associated files, or that could be merged into more common tags.