What I do is map this to a mouse button. I have a Logitech Performance Mouse MX.
As you can see from the picture below, I've mapped the (otherwise useless) 'Zoom Button' to 'Desktop'.
Really handy for seeing if the file you're looking for is on the Desktop. You can even manipulate files if you keep the assigned mouse button held in (i.e. you could grab some files and put them in the Trash; or you could attach a document to a Gmail email; etc.)
NB: I just noticed that you were already aware of this technique via a touchpad swipe. Nonetheless, I'll leave it up here as an answer as mapping it to a mouse button is something I can't live without.
The other way to do this in a more 'permanent' fashion would be to use the ⌥⌘H combo. This, too, can be mapped to a mouse button as you can see below: