7

After installing MacOS Catalina 10.15 Beta (19A471t) there is something taking up 90GB of my Disk. I tried scanning with Daisy Disk but wasn't able to identify what those files are.

How can I find out this folder?

About this Mac SS

  • Hey Alexandre! What does clicking Manage reveal? Go to Documents-> File Browser and wait for sometime to get the app to reveal sizes of each folder. – ankii Jun 17 at 18:12
  • I'm not exactly sure how we can help you. I think it would be best for you to file a bug report with Apple and see what they say. – grooveplex Jun 17 at 18:33
  • You could try Disk Inventory X if Daisy Disk isn't clear. Post a picture of the results. – lx07 Jun 17 at 20:13
  • 1
    Welcome to Ask Different :) Also, keep in mind that macOS Catalina is currently under developer beta, and unexpected behavior like this may not be surprising. – Nimesh Neema Jun 18 at 1:49
  • 1
    Maybe it could be related to local TimeMachine backups: forums.developer.apple.com/thread/117223 – MrCraftCod Jun 21 at 14:47
5

I cracked the code!

Preface

This seems to have worked fine for me, but there are no guarantees. DO NOT PROCEED WITHOUT A BACKUP OF YOUR DATA. Then again, you know that, you voluntarily installed a beta operating system. 😜

The root of this issue seems to be a failure in deleting APFS snapshots made by Time Machine. Under healthy operation, old snapshots are deleted as necessary whenever new disk space is required. However, this process failed, as we'll see below.

Symptoms

  1. Deleting files doesn't increase free space on the disk.
  2. Daisy Disk reports a large "hidden space". Deleting a file of X bytes would increase the size of this "hidden space" by X bytes.

First try thinning

Firstly, I would try to manually thin out the Time Machine snapshots. This requests Time Machine to automatically clean out enough snapshots to free a desired amount of space, in this case, 100 GB.

$ tmutil thinlocalsnapshots 100g 1

If this succeeded, it should say something like

Thinned local snapshots:
com.apple.TimeMachine.2019-06-21-141250.local

Listing some number of deleted snapshots. However, I doubt this will help. If the system could get this to work automatically, I don't see why manually invoking it would help. But YMMV, so it's worth a shot.

Manual snapshot deletion

After some number of snapshots were deleted, some space was freed up, but not much. the issue is that deleting any number of snapshots won't matter if even one snapshot exists holding onto the same data. In my case, there were two snapshots remaining that persisted even after thinning:

$ tmutil listlocalsnapshotdates
Snapshot dates for all disks:
2019-06-11-214224
2019-06-21-233121

I tried manually deleting these:

$ sudo tmutil deletelocalsnapshots 2019-06-21-233121
Deleted local snapshot '2019-06-21-233121'

$ sudo tmutil deletelocalsnapshots 2019-06-11-214224
Deleted local snapshot '2019-06-11-214224'

The response says they were deleted, but they actually weren't:

$ tmutil listlocalsnapshotdates
Snapshot dates for all disks:
2019-06-11-214224
2019-06-21-233121

At this point, I start going the nuclear route: directly using diskutil to delete the APFS snapshots, without Time Machine's blessing to do so. First, I listed the APFS snapshots to see their UUIDs:

$ sudo diskutil apfs listSnapshots /System/Volumes/Data
Snapshots for disk1s1 (2 found)
|
+-- BFD78F4F-99BB-4D5B-AE16-5367DC9C615E
|   Name:        com.apple.TimeMachine.2019-06-11-214224.local
|   XID:         10440723
|   Purgeable:   Yes
|
+-- 611E5357-8D10-4ABE-95F3-BE98C2DFCA3F
    Name:        com.apple.TimeMachine.2019-06-21-233121.local
    XID:         10492913
    Purgeable:   Yes
    NOTE:        This snapshot limits the minimum size of APFS Container disk1

Then I tried deleting them manually:

$ sudo diskutil apfs deleteSnapshot disk1s1 -uuid BFD78F4F-99BB-4D5B-AE16-5367DC9C615E
Deleting APFS Snapshot BFD78F4F-99BB-4D5B-AE16-5367DC9C615E "com.apple.TimeMachine.2019-06-11-214224.local" from APFS Volume disk1s1
Started APFS operation
Error: -69863: Insufficient privileges

Strangely, even though I'm using sudo to run the command as root, I'm told I have insufficient privileges. This might have something to do with the read-onliness of the system volume, or to do with it being the actively booted volume, but I don't know.

Going nuclear

I loaded up into recovery mode, and went to the command line. From there, I did a similar process to try to delete the snapshots. However, this required first unlocking and mounting the relevant volumes.

  1. Running diskutil list, I found that my data and system volumes were assigned the labels disk1s1 and disk2s5, respectively.

  2. I think (I don't remember precisely) unlocked the volumes with:

    -bash-3.2# diskutil apfs unlockVolume disk1s1
    Password:
    -bash-3.2# diskutil apfs unlockVolume disk1s5
    Password:
    
  3. Then I mounted them:

    -bash-3.2# diskutil mount disk1s1
    -bash-3.2# diskutil mount disk1s5
    
  4. I listed their snapshots:

    -bash-3.2# diskutil apfs listSnapshots disk2s1
    Snapshots for disk2s1 (2 found)
    |
    +-- Name: com.apple.TimeMachine.2019-06-11-214224.local
    |   XID:  10440723
    |
    +-- Name: com.apple.TimeMachine.2019-06-21-233121.local
        XID:  10492913
        NOTE: This snapshot sets the minimal allowed size of APFS Container disk2
    -bash-3.2# diskutil apfs listSnapshots disk2s5
    Snapshot for disk2s5 (2 found)
    |
    +-- Name: com.apple.TimeMachine.2019-06-11-214224.local
    |   XID:  187251
    |
    +-- Name: com.apple.TimeMachine.2019-06-21-233121.local
        XID:  10492911
        NOTE: This snapshot sets the minimal allowed size of APFS Container disk2
    
  5. I tried deleting the snapshots by their "XID", but that didn't seem to work for all of them, so I instead deleted them by name:

    -bash-3.2# diskutil apfs deleteSnapshot disk2s1 -name com.apple.TimeMachine.2019-06-11-214224.local
    Deleting APFS Snapshot XID 10440723 "com.apple.TimeMachine.2019-06-11-214224.local" from APFS Volume disk2s1
    Started APFS operation
    Finished APFS operation
    
    -bash-3.2# diskutil apfs deleteSnapshot disk2s1 -name com.apple.TimeMachine.2019-06-21-233121.local
    Deleting APFS Snapshot XID 10492913 "com.apple.TimeMachine.2019-06-21-233121.local" from APFS Volume disk2s1
    Started APFS operation
    Finished APFS operation
    
    -bash-3.2# diskutil apfs deleteSnapshot disk2s5 -name com.apple.TimeMachine.2019-06-11-214224.local
    Deleting APFS Snapshot XID 187251 "com.apple.TimeMachine.2019-06-11-214224.local" from APFS Volume disk2s5
    Started APFS operation
    Finished APFS operation
    
    -bash-3.2# diskutil apfs deleteSnapshot disk2s5 -name com.apple.TimeMachine.2019-06-21-233121.local
    Deleting APFS Snapshot XID 10492911 "com.apple.TimeMachine.2019-06-21-233121.local" from APFS Volume disk2s5
    Started APFS operation
    Finished APFS operation
    
  6. Once this was done, I confirmed that the snapshots were gone:

    -bash-3.2# diskutil apfs listSnapshots disk2s1
    No snapshots for disk2s1
    
    -bash-3.2# diskutil apfs listSnapshots disk2s5
    No snapshots for disk2s5
    
  7. And I confirmed that the space was in fact freed with diskutil apfs list.

  8. I restarted, and my Mac was back to normal. The space of all of the files I previously deleted is now visible and available.

And they lived happily ever after...

  • 2
    Holy cow, thank you so much! This has been maddening not knowing why deleting so many large files with DaisyDisk wasn't freeing up the space it should. – CamLeng Jun 22 at 17:47
  • Dude you're freaking insane — TO ANYONE UPDATING CATALINA WITHOUT CHECKING FOR STORAGE, FOLLOW THIS SAVIOUR'S INSTRUCTIONS (Alternatively don't be like me) — this helped so much. The problem is that you can't access many cl utilities because the OS is kinda broke. This helps a lot, the problem when you update without storage is that fact that the time machine bug kicks in resulting in an infinite loop. You're going to have to go and delete the time machine swapfile for your SSD so that that root os actually bothers to check how much storage is left. – aaaakshat Oct 10 at 8:11
2

@Alexander @Mane Manero As the tmutil usages shown:

Usage: tmutil deletelocalsnapshots [<mount_point> | <snapshot_date>]

You should feel free to delete snapshots using this command without sudo and recovery mode:

➜  ~ tmutil deletelocalsnapshots /System/Volumes/Data
Deleted 5 Time Machine local snapshots for volume group containing disk '/System/Volumes/Data'
com.apple.TimeMachine.2019-10-09-234112.local
com.apple.TimeMachine.2019-10-09-200334.local
com.apple.TimeMachine.2019-10-09-220434.local
com.apple.TimeMachine.2019-10-09-210802.local
com.apple.TimeMachine.2019-10-10-162541.local
1

I have many Apps on my system but my ~/Library folder tales up almost 100 GB. I suggest you check your library folder for the size. recall there are 2 library folders. one in your home directory and one on the same level as users or Application folder. Make sure you are showing hidden items then select your Library folder in the Finder and "Get Info"

1

It's worth saying that MacOS's Storage breakdown has traditionally been unreliable. As you're running beta software, make sure you report it to Apple.

Is the Finder reporting a similar amount of used/free space? If you have backups of iOS devices, like phones and iPads, these can take up large amounts of space. They used to be managed in iTunes, but this will have changed in Catalina.

1

Did you tell the installer to go ahead and copy any of your user data to the newly installed system? If yes, it's probably worth checking for expanded sparse files or sparse bundles/disk images. - Depending on the wisdom of the copying program, sparse files or bundles might have gotten expanded to their full glory, and take up "real" disk space now. Good candidates for sparse files on MacOS are sparse disk images, where the "room for expansion" got reserved during creation, but wasn't supposed to be taking up any space until filled with data. A simple copy can expand it as well...

To identify large files to inspect, you could use find:

  • find / -type f -size +2G -ls would list every file exceeding a 2 GB threshold.
  • if you want the output sorted by the size of those files, just add sort, and tell it to take the 7th column and display the results largest first (head further limits the output to the largest 20 of the bunch, to keep you from getting spammed):

    find / -type f -size +2G -ls | sort -k7 -r |head -20

Given the amount of space you're wondering about, chances are that this search would pop up the culprit. If it's something that was copied over from another installation, you could compare the space the file "really" takes up on disk by running du -sh </path/to/file> for both files. If it's smaller on the source side, re-copy it with a tool that keeps sparse files intact.

To identify by directory:

If you're more of a visual person, you could use a graphic tool to display the space taken up by directories, and drill down from there:

  • Disk Inventory X has been around forever, is free, and still actively maintained, while
  • Daisy Disk offers a modern interface, and a "free trial".

Unfortunately (as you've experienced) those tools only give a vague idea where large files may lie in hiding, but those can often serve as starting point for the commandline-based examinations.

If you prefer to examine the directories from the command line, you can find the large ones using find and du:

  • find / -type d -exec du -sh {} \; |grep "G " ("G " needs to be entered as Gcrtl-v<tab> to search for G<tab>) will list out all directories which are at least one G in size. Note that this will sum up nested directories several times depending on their nesting level (it will sum up /usr, /usr/local /usr/local/bin /usr/local/etc without trying to optimize the process).

It'd be interesting to hear back from you when you've identified the culprit!

  • 1
    Hi Tatjana, thanks for such a considered answer! I did scan with Daisy Disk, and it shows this in a "hidden folder" that I cannot access. I found that every file I delete, the "hidden space" gets increased with same size. So I really cannot decrease amount of storage. – Mane Manero Jun 18 at 16:34
  • Hey @Tatjana +1 for the answer. Could you include your affiliation with the apps please? Just for precaution. – ankii Jun 18 at 17:44
  • @AlexandreLordelo Just to make sure - you're not talking about the filesystem representation of the "Trashcan" from which you could restore files "deleted" via the Finder? (That one is cleaned out by selecting "Empty Trash" in the Finder Menu). To take a closer look at the "hidden folder" (which one and where?), you should be able to use sudo prepended to the command in question, for example sudo ls -l <path to folder>. The password sudo will ask for is the one belonging to your (administrative) user account. – Tatjana Heuser Jun 18 at 17:46
  • @ankiiiiiii I prefer shell over most GUI apps, so my affiliation isn't even that of a customer or user, I'm afraid. Ages ago (in the late 80es) I occasionally would use xdu which rendered some graphical output from running the du command recursively and saving the output to a file, the result looking somewhat similar to Disk Inventory X. – Tatjana Heuser Jun 18 at 17:52
  • @AlexandreLordelo "I found that every file I delete, the "hidden space" gets increased with same size. " I'm having exactly the same problem. I deleted my StarCraft 2 installation (about 30 GB), which grew my hidden space by 30 GB. – Alexander - Reinstate Monica Jun 22 at 3:29
0

Trimming / did not work for me on Catalina, /System/Volumes/Data worked for me

Check you disk space:

MacBook-Pro-2152:~ derek$ df -h
Filesystem      Size   Used  Avail Capacity iused      ifree %iused  Mounted on
/dev/disk1s5   466Gi   14Gi   36Gi    28%  478195 4881974685    0%   /
devfs          227Ki  227Ki    0Bi   100%     789          0  100%   /dev
/dev/disk1s1   466Gi  414Gi   36Gi    92% 3238291 4879214589    0%   /System/Volumes/Data
/dev/disk1s4   466Gi  1.0Gi   36Gi     3%       1 4882452879    0%   /private/var/vm
map auto_home    0Bi    0Bi    0Bi   100%       0          0  100%   /System/Volumes/Data/home

Trim the snapshots by ~10Gb

MacBook-Pro-2152:~ derek$ sudo tmutil thinLocalSnapshots /System/Volumes/Data 10000000000 4
Thinned local snapshots:
com.apple.TimeMachine.2019-08-08-014223.local
com.apple.TimeMachine.2019-08-08-095414.local
com.apple.TimeMachine.2019-08-08-105804.local

Confirm the free space

MacBook-Pro-2152:~ derek$ df -h
Filesystem      Size   Used  Avail Capacity iused      ifree %iused  Mounted on
/dev/disk1s5   466Gi   14Gi   47Gi    23%  478195 4881974685    0%   /
devfs          227Ki  227Ki    0Bi   100%     789          0  100%   /dev
/dev/disk1s1   466Gi  403Gi   47Gi    90% 3238291 4879214589    0%   /System/Volumes/Data
/dev/disk1s4   466Gi  1.0Gi   47Gi     3%       1 4882452879    0%   /private/var/vm
map auto_home    0Bi    0Bi    0Bi   100%       0          0  100%   /System/Volumes/Data/home
0

Before attempting the nuclear option, I found an article that helped. Of note, trimming did not work for me (which is why I'm assuming we're hitting this problem).

Per this link, you can list your local backups:

tmutil listlocalsnapshots /System/Volumes/Data

The result was

$tmutil listlocalsnapshots /System/Volumes/Data     

Snapshots for volume group containing disk /System/Volumes/Data:
com.apple.TimeMachine.2019-08-31-171636.local
com.apple.TimeMachine.2019-08-31-184648.local
com.apple.TimeMachine.2019-09-01-073333.local
com.apple.TimeMachine.2019-09-01-083316.local
com.apple.TimeMachine.2019-09-01-100431.local
com.apple.TimeMachine.2019-09-01-110546.local
com.apple.TimeMachine.2019-09-01-120530.local

Then run tmutil deletelocalsnapshots with the date and number to delete each one. Do one a time:

tmutil deletelocalsnapshots 2019-08-31-184648

tmutil deletelocalsnapshots 2019-08-31-184648

tmutil deletelocalsnapshots 2019-09-01-073333

...

After this, it took about 2 minutes and my computer reported 50GB free.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .