How to merge folders based on file content rather than filename

Question

I have multiple folders containing thousands of deletion-recovered files from multiple recovery scans.

As per usual with deletion recovery, the recovered files have different names than the originals, and different names from one recovery scan to another.

So, short of writing a custom program, is there a tool (or product) that can compare files by content and ignore the filenames?

Ideally, it would allow deletion of recovered files when they content-match files in an "original" directory tree (e.g. iTunes library folders), and, allow merge of two folders based on file contents rather than filenames.

Answer 1

Why wouldn't you have a script to loop over all the files and rename them with the value of a hash based on the file contents.

md5 file.png 
MD5 (file.png) = d5438a3187b73fa362dceb269e158908
crc32 file.png 
87a1ffdd
shasum -a 256 file.png 
34d126a6990b59dd45a4d7d19e471943569060bc54cb3b42b3d0be1934f16f11  file.png

I've listed these in order of fastest to slowest on my Mac - you should test for a few files if you can't just let the script run overnight and do care about performance of the script.

Then you could merge based on file name since the same content would be the same name, barring any collisions in the space of the hash.

• When is CRC more appropriate to use than MD5/SHA1?