Every time I try to open the .dmg to install XQuartz, a window pops up saying
the following disk images couldn't be opened and the reason shown is
not recognized. I don't know if I need another application to open it or if the file is corrupted, or if it's some other problem entirely.
I'd say the file downloaded is corrupted, you should be able to open the .dmg by simply double clicking it (I downloaded XQuartz a couple of days ago and could open the .dmg without a problem).
Unfortunately, the download page doesn't provide a checksum to check against, so I'll provide the one I generated:
This checksum uniquely identifies the XQuarz .dmg.
You can generate a checksum of XQuartz on your Mac like this (from http://support.apple.com/kb/HT1652):