2

As I understand, HFS+ and APFS volumes don't need to be defragmented, as files will automatically defragmented by the OS, but I'd like to leave a process running in the background that would do this to all files of the disk. Is there a way to check which files have more fragments or get a list of all files of the disk?

  • I'm not sure why you think files are relocated when they are opened - do you have any source for that? For APFS you can whether check whether automatic defragmentation is enabled with (for example) diskutil apfs defrag disk1s1 status. It is disabled by default but it isn't clear what (if anything) enabling it does. – lx07 Oct 9 at 21:49
  • Sorry. That seems to be a common "urban legend". The OS does some magic with files that have 8 or more fragments or are under 20 MB, also there is the "hot file adaptive clustering", but I couldn't find much information on how, exactly, the OS deals with that. Judging from an experience I had last night, when moving large files between disks, some heavy lifting is done during shutdown (as it took many hours). – rbanffy Oct 10 at 10:34
  • 1
    @lx07 There is some mention of this behavior here: osxbook.com/book/bonus/misc/optimizations. The page has been there for a very long time, so I can't tell how up-to-date it is. – rbanffy Oct 10 at 10:47
  • 2
    @lx07 You can read my summary of how it works in this answer: apple.stackexchange.com/questions/344122/… – jksoegaard Oct 10 at 11:09
  • 1
    So on HFS+ some files are actually defragmented on open, that's not an urban legend. However, it only happens for files <20 MB that are not busy, not read-only, unchanged for at least 60 seconds and has more than 8 extents. You're not typically going to be creating many of those on a daily basis (unless you have an almost full disk, and weird access patterns). – jksoegaard Oct 10 at 11:10
1

It is correct that on HFS+ volumes a technique named "On-the-fly" automatically defragments certain files when they're opened. However, it only applies for files that are less than 20 MB in size, are not busy, not read-only, have been unchanged for at least 60 seconds and has more than 8 extents (i.e. "fragments"). Typically those files will occupy a very small percentage of your hard drive.

In order to get statistics and list of fragmented files on HFS+ volumes, you'll want to use a tool like "hfsdebug" (free). It offers both listing of all fragmented files, as well as various statistics such as top lists of most fragmented files.

In order to get the list of all fragmented files that you seek, run the following command in Terminal.app after installing hfsdebug:

hfsdebug -f /

You can choose to see only the top 10 files like this:

hfsdebug -t 10 -f /

A newer tool named fileXray (79$) also exists that gives you much of the same features, and plenty more. As noted it is commercial software that requires you to purchase a license - note that it does not have a GUI, but has a textual interface similar to hfsdebug.

You can use fileXray to for example check if a file is fragmented or not:

fileXray filename

This will output a list of extents for the file "filename". If the file has multiple extents it is in theory fragmented. If the startBlock + blockCount of an extent lines up with the startBlock of the next extent, then in practice it doesn't matter much.

A simple way to do the actual defragmentation is to simply copy the file. Ensure that you have enough (non-fragmented) disk space in advance, and then copy the file to a new name on the same disk. The new file should be less fragmented (or not fragmented at all) - you can check that with the above mentioned command. After copying, you can remove the old file and rename the new file to match.

UPDATE: In the comments, you have expressed that you needed a free solution, and cannot use fileXray because it costs 75$. In that case you could use SleuthKit from HomeBrew instead - you'll find the ifind command from SleuthKit can list the number of extents as well. Another possibility is hfsinspect by Adam Knight, which can also list the extents.

  • The author retired hfsdebug after considering it obsolete and reporting incorrect information. It's a shame Apple doesn't have a similar tool (that also works on APFS). – rbanffy Oct 10 at 14:02
  • @rbanffy If you have a copy, you can still run it though - but yeah, fileXray is the new tool. – jksoegaard Oct 10 at 14:29
0

According to, https://osxbook.com/book/bonus/misc/optimizations/#FIVE files are defragmented on open if all the conditions below are true:

  • the file is less than 20 MB in size
  • the file is not already busy
  • the file is not read-only
  • the file has more than eight extents
  • the system has been up for at least three minutes

If that still holds, a simple way to defragment a folder would be, from the terminal:

find . -type f -exec head {} > /dev/null \;

As this will read the start of every file within that folder and do nothing with it. Since it requires opening the file, it should trigger the defragmentation and hot file clustering behaviors for all writable files below 20 MBs.

I can't really tell whether it works.

  • This is exactly what I wrote in my answer to you earlier - there’s no need to repeat the same answer again. If you want to add a link, just edit and add it. – jksoegaard Oct 10 at 16:46
  • 1
    Regarding the last part of this answer - this has nothing to do with hot file clustering. That works completely different, and only for a narrower set of files that are even smaller (<10MB). I wrote in my answer how to see if a command like yours work... – jksoegaard Oct 10 at 16:49
  • Good point. It should only trigger the automatic defragmentation. HFAC is a separate process - it'll only be triggered here if the extra open fools macOS and pushes the file over macOS's threshold (is APFS's source open?) for hot files. Your solution of copying is better, as it's guaranteed to defragment all files (including large ones that macOS doesn't even try to defragment) at the expense of not returning control to the terminal until the process is complete. If we have very high fragmentation, it may actually fragment more if the original is less fragmented than what the allocator finds. – rbanffy Oct 11 at 9:19
  • No, it won't fragment more. As I wrote in my answer, after copying you can check with the command to ensure that the file is less fragmented than before. If it is even more fragmented, then you undo (i.e. just delete the new file). – jksoegaard Oct 11 at 13:00
  • 1
    And there's no "at the expense of not returning control to the terminal" - if you want to return control to the terminal at once, then that's very easy - just put the process in the background. If you're running the standard shell, just put & at the end of the command string, and you'll run it in the background. If you want to run this automatically for every file on the disk, just create a small shell script. It could for example use find to trigger a copy/check/move for each file on the drive. That shell script could then be run from cron periodically, or just manually in the background. – jksoegaard Oct 11 at 13:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .