I'm comparing two large directories on two different external FireWire 800 disks using
diff -r /path/to/directory1 /path/to/directory2
The size of the directory (and its subdirectories) on one FireWire disk is 118.2 GB for 30,000 items and 119.56 GB for 30,031 items.
I was surprised about the many differences reported in the output, like
Binary files /path/to/directory1/with/subdirectory/file_xyz and /path/to/directory2/with/subdirectory/file_xyz differ
and started comparing some of them individually. When I compare them with
diff /path/to/directory1/with/subdirectory/with/subdirectory/file_xyz /path/to/directory2/with/subdirectory/with/subdirectory/file_xyz
diff /path/to/directory1/with/subdirectory/ /path/to/directory2/with/subdirectory/
diff doesn't report any differences between these files or directories.
What could be a reason why the "large comparison" reports so many differences (or seemingly fails), while the smaller chunk comparison doesn't?
Edits since original post:
- The version of
diffI'm using is GNU diffutils 2.8.1.
- Just a wild guess here, but could it have something to do with the fact that these directories are on external hard drives that could experience some sort of timeout?
- I ran another comparison and was again presented with lots of differences between those directories. I then set the system preference's Energy Saver to not set the display to sleep for 1 hour, because I had measured with
time diff -r /path/to/directory1 /path/to/directory2that it takes between 45 and 50 minutes for the diff to finish. My hard drives and the Mac never go to sleep.
I then unmounted both drives and remounted them. Ran the diff again and voilà no differences found apart from one file. I manually compared that one reported differing file and found it to be identical.
This seems to confirm what I found by comparing smaller chunks before. It also seems to confirm that there is something going on with a timeout as previously suspected, but I still wonder that if a FireWire drive would become unavailable or whatever, diff should not report a difference, but rather say "file not available" or "Only in /yada/yada/directory: file_xyz".
Interestingly, the diff before the remount took 28 minutes, after the remount it took almost 51 minutes.
In the light of that: What can I do to prevent something like that from happening?
Of course one could say to never set the display to go to sleep or whatever, but that hardly seems to solve the underlying issue. Something else must be going on, I just can't figure out what.
As an aside, on previous other occasions, after some time of inactivity on those FireWire hard drives I was trying to write to a file and got this *error code -50" message (mind you I didn't see any of that during the diff operation). I could always "resolve" the issue by unmounting and remounting the drives, but I believe there must be an entirely different solution to that:
Solutions like those presented here hardly seem to be tackling the underlying problem.