Delete a folder with a lot of files

Question

I'm not even sure how to ask this question properly, but let me try.

I'm on Mojave 10.14.

I found a directory in my trash can that is extremely slow to delete. When I try to go into the directory, finder shows spinning at the bottom and does not displays the files in it. I've waited well over 10 mins.

When I try to use "ls" cmd in the terminal, it also takes over 10 mins and does not display anything.

When I try to empty the trash, it is EXTREMLY slow. The # of files deleted increases 1 per second (almost). I've waited well over 30 mins and it was still not finished.

How do I diagnose this problem and find a solution?

Update 1: Filesystem is APFS and the directory has more than 12,000,000 small json files. Update 2: Finally counted the # of files and total size of the directory. The directory has 22,203,359 files and 27.59GB.

Answer 1

unix.stackexchange.com once told me that rsync is the fastes way to delete many files:

mkdir empty_dir
rsync -a --delete empty_dir/    yourdirectory/

Maybe this helps here.

Answer 2

When I try to use "ls" cmd in the terminal, it also takes over 10 mins and does not display anything.

1. This is not normal. Something is amiss. I'd see the most likely cause as either a bad filesystem or a failing drive. While surprising, it could be a bad spot on the drive just where the directory is. In the case of a failing drive, macOS will try multiple reads to see if it can retrieve the data.

   /Applications/Utilities/Disk Utility

goto the First Aid tab. Should you find problems, you will have to boot in the recovery manager. command + r at boot time before the beep. click on the utilities tab and select disk utility.

High Sierra's Disk Utility

Yosemite's Disk Utility

2. Try a safe boot. It's always a good idea to try safe mode when strange things happen.

   Shutdown your machine. Hold down the shift key. Poweron. The boot up will take longer than normal because the filesystem on the startup drive is being checked and repaired as needed. All about safe mode including what features and apps safe boot leaves out. Safe boot uses a software driver instead of using your machines video hardware. http://support.apple.com/kb/ht1455

Answer 3

Deleting a lot of files from Trash via Finder/Empty Trash can be very slow sometimes. It might be easier/faster to run

rm -r ~/.Trash

in Terminal (at least it won't partially block the UI while doing so).

PS: If the files to be deleted are on an external drive you need to delete them from /Volumes/VOLNAME/.Trash instead.

Answer 4

22 million files at 1 millisecond each will take over 6 hours.

I've no idea if that's a realistic estimate for the unit time, but the Finder is traditionally not the most efficient at file operations.

And I wouldn't trust its 'progress' dialog, either. https://xkcd.com/612/

You should leave the process to run for as long as it takes.

Answer 5

I saw some suggestions for rm -rf ./dirname to delete a lot of files in the terminal. It appears nothing is happening, but it's because you won't see what it's doing unless you use the "verbose" flag of -v.

So try:

rm -rfv ./YOUR_DIR_NAME_YOU_WANT_TO_DELETE

Then you can see the files being deleted as they go. Since there are a large number of them, it may take a while. Note that verbose mode might slow it down in the end, but I prefer to see that it's still working.

Answer 6

Even "simple" tools like rm are actually several thousand lines of code and full of complicated features that slow them down. In addition, almost all filesystems are not designed to hold large numbers of files in a single directory (10,000 is a sensible upper limit or 1,000 if using a GUI file browser).

Here's a simple script that uses low level UNIX APIs (unlink() and rmdir()) to delete each individual item in the directory, using the nftw() API to scan through the directory.

Simply save it as a text file, make it executable, and run it. I'd expect it to be very fast even with 22 million files.

#!/usr/bin/swift

import Foundation

let path = "/Path/To/Folder"

nftw(path, { child, _, type, _ in
    if (type == FTW_F) { // file
      unlink(child)
    } else if (type == FTW_DP) { // directory "where we have visited children"
      rmdir(child)
    } else if (type == FTW_SL || type == FTW_SLN) { // symlink
      unlink(child)
    } else { // anything else, assume some kind of permission issue or similar
      print("Cannot delete \(String(cString: child!))")
      return 1 // error
    }
    return 0 // no error
  },
  42, // how many file descriptors should we use?
  FTW_PHYS | FTW_DEPTH // do not follow symlinks and do a "depth first" scan (files are provided before their parent dir)
)