After installing MacOS Catalina 10.15 Beta (19A471t) there is something taking up 90GB of my Disk. I tried scanning with Daisy Disk but wasn't able to identify what those files are.
How can I find out this folder?
8 Answers
I cracked the code!
Preface
This seems to have worked fine for me, but there are no guarantees. DO NOT PROCEED WITHOUT A BACKUP OF YOUR DATA. Then again, you know that, you voluntarily installed a beta operating system. 😜
The root of this issue seems to be a failure in deleting APFS snapshots made by Time Machine. Under healthy operation, old snapshots are deleted as necessary whenever new disk space is required. However, this process failed, as we'll see below.
Symptoms
- Deleting files doesn't increase free space on the disk.
- Daisy Disk reports a large "hidden space". Deleting a file of X bytes would increase the size of this "hidden space" by X bytes.
First try thinning
Firstly, I would try to manually thin out the Time Machine snapshots. This requests Time Machine to automatically clean out enough snapshots to free a desired amount of space, in this case, 100 GB.
$ tmutil thinlocalsnapshots 100g 1
If this succeeded, it should say something like
Thinned local snapshots:
com.apple.TimeMachine.2019-06-21-141250.local
Listing some number of deleted snapshots. However, I doubt this will help. If the system could get this to work automatically, I don't see why manually invoking it would help. But YMMV, so it's worth a shot.
Manual snapshot deletion
After some number of snapshots were deleted, some space was freed up, but not much. the issue is that deleting any number of snapshots won't matter if even one snapshot exists holding onto the same data. In my case, there were two snapshots remaining that persisted even after thinning:
$ tmutil listlocalsnapshotdates
Snapshot dates for all disks:
2019-06-11-214224
2019-06-21-233121
I tried manually deleting these:
$ sudo tmutil deletelocalsnapshots 2019-06-21-233121
Deleted local snapshot '2019-06-21-233121'
$ sudo tmutil deletelocalsnapshots 2019-06-11-214224
Deleted local snapshot '2019-06-11-214224'
The response says they were deleted, but they actually weren't:
$ tmutil listlocalsnapshotdates
Snapshot dates for all disks:
2019-06-11-214224
2019-06-21-233121
At this point, I start going the nuclear route: directly using diskutil to delete the APFS snapshots, without Time Machine's blessing to do so. First, I listed the APFS snapshots to see their UUIDs:
$ sudo diskutil apfs listSnapshots /System/Volumes/Data
Snapshots for disk1s1 (2 found)
|
+-- BFD78F4F-99BB-4D5B-AE16-5367DC9C615E
| Name: com.apple.TimeMachine.2019-06-11-214224.local
| XID: 10440723
| Purgeable: Yes
|
+-- 611E5357-8D10-4ABE-95F3-BE98C2DFCA3F
Name: com.apple.TimeMachine.2019-06-21-233121.local
XID: 10492913
Purgeable: Yes
NOTE: This snapshot limits the minimum size of APFS Container disk1
Then I tried deleting them manually:
$ sudo diskutil apfs deleteSnapshot disk1s1 -uuid BFD78F4F-99BB-4D5B-AE16-5367DC9C615E
Deleting APFS Snapshot BFD78F4F-99BB-4D5B-AE16-5367DC9C615E "com.apple.TimeMachine.2019-06-11-214224.local" from APFS Volume disk1s1
Started APFS operation
Error: -69863: Insufficient privileges
Strangely, even though I'm using sudo to run the command as root, I'm told I have insufficient privileges. This might have something to do with the read-onliness of the system volume, or to do with it being the actively booted volume, but I don't know.
Going nuclear
I loaded up into recovery mode, and went to the command line. From there, I did a similar process to try to delete the snapshots. However, this required first unlocking and mounting the relevant volumes.
Running
diskutil list, I found that my data and system volumes were assigned the labelsdisk1s1anddisk2s5, respectively.I think (I don't remember precisely) unlocked the volumes with:
-bash-3.2# diskutil apfs unlockVolume disk1s1 Password: -bash-3.2# diskutil apfs unlockVolume disk1s5 Password:Then I mounted them:
-bash-3.2# diskutil mount disk1s1 -bash-3.2# diskutil mount disk1s5I listed their snapshots:
-bash-3.2# diskutil apfs listSnapshots disk2s1 Snapshots for disk2s1 (2 found) | +-- Name: com.apple.TimeMachine.2019-06-11-214224.local | XID: 10440723 | +-- Name: com.apple.TimeMachine.2019-06-21-233121.local XID: 10492913 NOTE: This snapshot sets the minimal allowed size of APFS Container disk2 -bash-3.2# diskutil apfs listSnapshots disk2s5 Snapshot for disk2s5 (2 found) | +-- Name: com.apple.TimeMachine.2019-06-11-214224.local | XID: 187251 | +-- Name: com.apple.TimeMachine.2019-06-21-233121.local XID: 10492911 NOTE: This snapshot sets the minimal allowed size of APFS Container disk2I tried deleting the snapshots by their "XID", but that didn't seem to work for all of them, so I instead deleted them by name:
-bash-3.2# diskutil apfs deleteSnapshot disk2s1 -name com.apple.TimeMachine.2019-06-11-214224.local Deleting APFS Snapshot XID 10440723 "com.apple.TimeMachine.2019-06-11-214224.local" from APFS Volume disk2s1 Started APFS operation Finished APFS operation -bash-3.2# diskutil apfs deleteSnapshot disk2s1 -name com.apple.TimeMachine.2019-06-21-233121.local Deleting APFS Snapshot XID 10492913 "com.apple.TimeMachine.2019-06-21-233121.local" from APFS Volume disk2s1 Started APFS operation Finished APFS operation -bash-3.2# diskutil apfs deleteSnapshot disk2s5 -name com.apple.TimeMachine.2019-06-11-214224.local Deleting APFS Snapshot XID 187251 "com.apple.TimeMachine.2019-06-11-214224.local" from APFS Volume disk2s5 Started APFS operation Finished APFS operation -bash-3.2# diskutil apfs deleteSnapshot disk2s5 -name com.apple.TimeMachine.2019-06-21-233121.local Deleting APFS Snapshot XID 10492911 "com.apple.TimeMachine.2019-06-21-233121.local" from APFS Volume disk2s5 Started APFS operation Finished APFS operationOnce this was done, I confirmed that the snapshots were gone:
-bash-3.2# diskutil apfs listSnapshots disk2s1 No snapshots for disk2s1 -bash-3.2# diskutil apfs listSnapshots disk2s5 No snapshots for disk2s5And I confirmed that the space was in fact freed with
diskutil apfs list.I restarted, and my Mac was back to normal. The space of all of the files I previously deleted is now visible and available.
And they lived happily ever after...
-
3Holy cow, thank you so much! This has been maddening not knowing why deleting so many large files with DaisyDisk wasn't freeing up the space it should.– camlengJun 22, 2019 at 17:47
-
Dude you're freaking insane — TO ANYONE UPDATING CATALINA WITHOUT CHECKING FOR STORAGE, FOLLOW THIS SAVIOUR'S INSTRUCTIONS (Alternatively don't be like me) — this helped so much. The problem is that you can't access many cl utilities because the OS is kinda broke. This helps a lot, the problem when you update without storage is that fact that the time machine bug kicks in resulting in an infinite loop. You're going to have to go and delete the time machine swapfile for your SSD so that that root os actually bothers to check how much storage is left. Oct 10, 2019 at 8:11
-
works if you have snapshots, unfortunatly that's not the case for a lot of us :/ Jan 26, 2020 at 16:14
Did you tell the installer to go ahead and copy any of your user data to the newly installed system? If yes, it's probably worth checking for expanded sparse files or sparse bundles/disk images. - Depending on the wisdom of the copying program, sparse files or bundles might have gotten expanded to their full glory, and take up "real" disk space now. Good candidates for sparse files on MacOS are sparse disk images, where the "room for expansion" got reserved during creation, but wasn't supposed to be taking up any space until filled with data. A simple copy can expand it as well...
To identify large files to inspect, you could use find:
find / -type f -size +2G -lswould list every file exceeding a 2 GB threshold.if you want the output sorted by the size of those files, just add sort, and tell it to take the 7th column and display the results largest first (
headfurther limits the output to the largest 20 of the bunch, to keep you from getting spammed):find / -type f -size +2G -ls | sort -k7 -r |head -20
Given the amount of space you're wondering about, chances are that this search would pop up the culprit. If it's something that was copied over from another installation, you could compare the space the file "really" takes up on disk by running du -sh </path/to/file> for both files. If it's smaller on the source side, re-copy it with a tool that keeps sparse files intact.
To identify by directory:
If you're more of a visual person, you could use a graphic tool to display the space taken up by directories, and drill down from there:
- Disk Inventory X has been around forever, is free, and still actively maintained, while
- Daisy Disk offers a modern interface, and a "free trial".
Unfortunately (as you've experienced) those tools only give a vague idea where large files may lie in hiding, but those can often serve as starting point for the commandline-based examinations.
If you prefer to examine the directories from the command line, you can find the large ones using find and du:
find / -type d -exec du -sh {} \; |grep "G "("G "needs to be entered as Gcrtl-v<tab>to search forG<tab>) will list out all directories which are at least one G in size. Note that this will sum up nested directories several times depending on their nesting level (it will sum up /usr, /usr/local /usr/local/bin /usr/local/etc without trying to optimize the process).
It'd be interesting to hear back from you when you've identified the culprit!
-
1Hi Tatjana, thanks for such a considered answer! I did scan with Daisy Disk, and it shows this in a "hidden folder" that I cannot access. I found that every file I delete, the "hidden space" gets increased with same size. So I really cannot decrease amount of storage. Jun 18, 2019 at 16:34
-
Hey @Tatjana +1 for the answer. Could you include your affiliation with the apps please? Just for precaution.– ankiJun 18, 2019 at 17:44
-
@AlexandreLordelo Just to make sure - you're not talking about the filesystem representation of the "Trashcan" from which you could restore files "deleted" via the Finder? (That one is cleaned out by selecting "Empty Trash" in the Finder Menu). To take a closer look at the "hidden folder" (which one and where?), you should be able to use
sudoprepended to the command in question, for examplesudo ls -l <path to folder>. The passwordsudowill ask for is the one belonging to your (administrative) user account. Jun 18, 2019 at 17:46 -
@ankiiiiiii I prefer shell over most GUI apps, so my affiliation isn't even that of a customer or user, I'm afraid. Ages ago (in the late 80es) I occasionally would use
xduwhich rendered some graphical output from running theducommand recursively and saving the output to a file, the result looking somewhat similar toDisk Inventory X. Jun 18, 2019 at 17:52 -
@AlexandreLordelo "I found that every file I delete, the "hidden space" gets increased with same size. " I'm having exactly the same problem. I deleted my StarCraft 2 installation (about 30 GB), which grew my hidden space by 30 GB. Jun 22, 2019 at 3:29
I have many Apps on my system but my ~/Library folder tales up almost 100 GB. I suggest you check your library folder for the size. recall there are 2 library folders. one in your home directory and one on the same level as users or Application folder. Make sure you are showing hidden items then select your Library folder in the Finder and "Get Info"
It's worth saying that MacOS's Storage breakdown has traditionally been unreliable. As you're running beta software, make sure you report it to Apple.
Is the Finder reporting a similar amount of used/free space? If you have backups of iOS devices, like phones and iPads, these can take up large amounts of space. They used to be managed in iTunes, but this will have changed in Catalina.
@Alexander @Mane Manero
As the tmutil usages shown:
Usage: tmutil deletelocalsnapshots [<mount_point> | <snapshot_date>]
You should feel free to delete snapshots using this command without sudo and recovery mode:
➜ ~ tmutil deletelocalsnapshots /System/Volumes/Data
Deleted 5 Time Machine local snapshots for volume group containing disk '/System/Volumes/Data'
com.apple.TimeMachine.2019-10-09-234112.local
com.apple.TimeMachine.2019-10-09-200334.local
com.apple.TimeMachine.2019-10-09-220434.local
com.apple.TimeMachine.2019-10-09-210802.local
com.apple.TimeMachine.2019-10-10-162541.local
Found... macOS Catalina File System Changes: Auto Generated Multiple Files as swapfile (AnyNumber) of 1.07GB under HD > Private > var > VM
You can view hidden files in Finder by pressing : ⌘ Command ⇧ Shift .
Trimming / did not work for me on Catalina, /System/Volumes/Data worked for me
Check you disk space:
MacBook-Pro-2152:~ derek$ df -h
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk1s5 466Gi 14Gi 36Gi 28% 478195 4881974685 0% /
devfs 227Ki 227Ki 0Bi 100% 789 0 100% /dev
/dev/disk1s1 466Gi 414Gi 36Gi 92% 3238291 4879214589 0% /System/Volumes/Data
/dev/disk1s4 466Gi 1.0Gi 36Gi 3% 1 4882452879 0% /private/var/vm
map auto_home 0Bi 0Bi 0Bi 100% 0 0 100% /System/Volumes/Data/home
Trim the snapshots by ~10Gb
MacBook-Pro-2152:~ derek$ sudo tmutil thinLocalSnapshots /System/Volumes/Data 10000000000 4
Thinned local snapshots:
com.apple.TimeMachine.2019-08-08-014223.local
com.apple.TimeMachine.2019-08-08-095414.local
com.apple.TimeMachine.2019-08-08-105804.local
Confirm the free space
MacBook-Pro-2152:~ derek$ df -h
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk1s5 466Gi 14Gi 47Gi 23% 478195 4881974685 0% /
devfs 227Ki 227Ki 0Bi 100% 789 0 100% /dev
/dev/disk1s1 466Gi 403Gi 47Gi 90% 3238291 4879214589 0% /System/Volumes/Data
/dev/disk1s4 466Gi 1.0Gi 47Gi 3% 1 4882452879 0% /private/var/vm
map auto_home 0Bi 0Bi 0Bi 100% 0 0 100% /System/Volumes/Data/home
Before attempting the nuclear option, I found an article that helped. Of note, trimming did not work for me (which is why I'm assuming we're hitting this problem).
Per this link, you can list your local backups:
tmutil listlocalsnapshots /System/Volumes/Data
The result was
$tmutil listlocalsnapshots /System/Volumes/Data
Snapshots for volume group containing disk /System/Volumes/Data:
com.apple.TimeMachine.2019-08-31-171636.local
com.apple.TimeMachine.2019-08-31-184648.local
com.apple.TimeMachine.2019-09-01-073333.local
com.apple.TimeMachine.2019-09-01-083316.local
com.apple.TimeMachine.2019-09-01-100431.local
com.apple.TimeMachine.2019-09-01-110546.local
com.apple.TimeMachine.2019-09-01-120530.local
Then run tmutil deletelocalsnapshots with the date and number to delete each one. Do one a time:
tmutil deletelocalsnapshots 2019-08-31-184648
tmutil deletelocalsnapshots 2019-08-31-184648
tmutil deletelocalsnapshots 2019-09-01-073333
...
After this, it took about 2 minutes and my computer reported 50GB free.
-
If you just want to delete all of them, you might save some time by looping over the dates like so:
for snapshot in $(tmutil listlocalsnapshotdates | grep 20); do sudo tmutil deletelocalsnapshots $snapshot; doneApr 2, 2020 at 15:24
Managereveal? Go to Documents-> File Browser and wait for sometime to get the app to reveal sizes of each folder.