Diagnosing runaway system storage when OmniDiskSweeper fails

Question

Every few days now, I clear 1-2 GB from my hard drive and still get this alert of low disk space on the desktop:

Your disk is almost full. Save space by optimising storage

And in the Terminal:

Your disk space is critically low. Older terminal scrollback contents may be automatically discarded to conserve VM backing store.:

I click on Manage from the desktop and see that system storage is taking 84 GB on a 128 GB SSD drive:

OmniDiskSweeper fails to find the large files: the total of files is about 43 GB out of 110 GB used:

The alternative is to keep moving files to an external drive and clear disk space until there is nothing left and I reinstall.

How can I find the culprit of runaway system storage?

Note: A few years ago I asked Terminal command to identify and clear runaway system storage and solved it with OmniDiskSweeper. Now I have the same issue and cannot find the culprit. I also know about How can I figure out what's slowly eating my drive space? and tried some of the tools there.

Update

I ran this du command on /Volumes/Macintosh HD, and here is the list of folders with over 2 GB. Colossus is the name of my hard drive. Nothing catches my eye from this list:

$ sudo du -h | sort -h
...
2.0G    ./Volumes/Colossus/Library/Application Support/MobileSync
2.0G    ./Volumes/Colossus/Library/Application Support/MobileSync/Backup
2.1G    ./Users/mmorin/Library/Application Support/MobileSync
2.1G    ./Users/mmorin/Library/Application Support/MobileSync/Backup
2.1G    ./Users/mmorin/Music/iTunes/iTunes Media
2.2G    ./Applications/iMovie.app
2.2G    ./Applications/iMovie.app/Contents
2.2G    ./Users/mmorin/reference
2.3G    ./Users/mmorin/Music
2.3G    ./Users/mmorin/Music/iTunes
2.3G    ./private/var/db
2.4G    ./Users/mmorin/Library/Caches/Homebrew
2.5G    ./Volumes/Colossus/Library/Containers
2.6G    ./usr/local/texlive/2020/texmf-dist/fonts
2.7G    ./usr/local/texlive/2020/texmf-dist/doc
3.0G    ./private/var/folders
3.0G    ./private/var/vm
3.2G    ./Library/Application Support/Adobe
3.2G    ./Users/mmorin/Maildir/Cantab/Deleted Items
3.2G    ./Users/mmorin/Maildir/Cantab/Deleted Items/cur
3.2G    ./Volumes/Colossus/Library/Messages
3.6G    ./Users/mmorin/Library/Caches
4.4G    ./Library/Application Support
4.4G    ./Volumes/Colossus/Library/Application Support
4.5G    ./Users/mmorin/Maildir/Cantab
5.2G    ./usr/local/Cellar
5.4G    ./Users/mmorin/Library/Application Support
6.2G    ./Users/mmorin/Maildir
6.2G    ./usr/local/texlive/2020/texmf-dist
6.5G    ./usr/local/texlive
6.5G    ./usr/local/texlive/2020
6.7G    ./Library
6.8G    ./System/Library
6.9G    ./System
8.9G    ./private/var
9.0G    ./private
9.1G    ./.Spotlight-V100
9.1G    ./.Spotlight-V100/Store-V2
9.1G    ./.Spotlight-V100/Store-V2/CA9AE856-FB94-4765-9028-38E04BEC9683
9.7G    ./Applications
 13G    ./Users/mmorin/Library
 13G    ./Volumes/Colossus/Library
 15G    ./Volumes
 15G    ./Volumes/Colossus
 16G    ./usr
 16G    ./usr/local
 29G    ./Users
 29G    ./Users/mmorin

Disk Utility on the container disk shows this:

Answer 1

a short (for sure improvable) introduction to regaining storage space on an internal drive

1. you should use the manage-function Apple provides to get a rough impression what’s occupying storage. Images, videos, music and podcasts are a good place to start freeing up space - Apple thinks so too. 🤓

2. To get a detailed impression of your system, start with opening a terminal and run df -h. The -h flag returns a ‘human readable form’ meaning return in the familiar megabyte/gigabyte format. You should see something like this (it is somewhat redundant if you already followed the first point in this list):

   $ df -h
   Filesystem Size Used Avail Use% Mounted on
   /dev/disk0s2 128G 122G 6G 95% / 
   

3. to determine where storage is occupied continue in the terminal and change to the root folder using cd / then run sudo du -sh * | sort -h . This will produce a list of file space usage, something like:

   3.2G    ./Users/your_user/Maildir/your_mailaccount01/Deleted Items
   4.4G    ./Library/Application Support
   4.4G    ./Volumes/Colossus/Library/Application Support
   4.5G    ./Users/your_user/Maildir/
   5.2G    ./usr/local/
   5.4G    ./Users/your_user/Library/Application Support
   6.2G    ./Users/your_user/Maildir
   6.5G    ./usr/local/texlive
   9.1G    ./.Spotlight-V100
   

this list might go on and on… In any case you should be able to pinpoint where to start to regain space on the drive.

4. In the above example

    3.2G    ./Users/your_user/Maildir/your_mailaccount01/Deleted Items
   

indicates that there are 3.2Gb of deleted mail items, which are still not removed from the drive. In the Mail app on your Mac, do one of the following:

• Choose Mailbox > Erase Deleted Items, then choose an account.
• In the Mail sidebar, Control-click a Bin mailbox, then choose Erase Deleted Items. The options differ slightly depending on the MacOS version - see Apple’s guide here. If you use another Mail client, please check their manual.

5. next stop in our example would be the TeXLive installation:

    6.5G    ./usr/local/texlive
   

TeX user group offers BasicTeX, a much smaller alternate TeX Distribution for users who do not want to install the full TeX Live. Depending on how TeX Live was installed, you simply drag the texlive-folder to the trash and empty it.

6. or if TeX (or any other program package) was installed via Homebrew use brew remove <package-name>.

7. next the Spotlight index on the example drive:

    9.1G    ./.Spotlight-V100
   

The index size depends of course on how many drives with how many files etc. you assigned to get indexed, but as a "rule of thumb" - if the Spotlight index occupies a few gigabytes something is off and a delete&reindex seems advisable:

• Open  > System Preferences, then click Spotlight , then the Privacy tab. Drag your internal disk to the list of locations that Spotlight is prevented from searching. Wait a bit. Then click the remove button (–) to remove it from the list and start the reindexing.

• Or use the terminal - type and run the command: sudo -i, enter your administrator password and press Enter. Your Terminal changes to root. Run the commands: mdutil -Ea and mdutil -ai off. Enter the command mdutil -ai on and press the Return. You will see a message saying that indexing is enabled.

8. Make sure Time Machine backups to external storage.

For sure this list is neither complete nor in order of importance, but hopefully clear and comprehensible for the few points it provides.

Answer 2

<rant>6.6GB is already critically low remaining space. You need 10-15% free space at minimum, but ironically even more if you only have a small disk. For a teeny 128GB drive, I would be looking to keep at least 20GB free at all times. [It's not really your fault. The disk is too small to be vaguely fit for purpose & all laptop makers ought to be ashamed of themselves for selling computers with such abysmally small storage in the first place. At the time, SSDs were expensive, so it was a cost-cutting exercise. If it's replaceable, replace it with at least a 512. </rant>

OK, more practically, look in Disk Utility & see how much 'Free' space there is compared to 'Available' & 'Purgeable'. Our computers are not vaguely comparable, but look at these 2 figues, in red. The difference is likely to be your actual problem. Running Time Machine might clear some of it, as it will clear your locally stored snapshot backlog.