7

I'm not even sure how to ask this question properly, but let me try.

I'm on Mojave 10.14.

I found a directory in my trash can that is extremely slow to delete. When I try to go into the directory, finder shows spinning at the bottom and does not displays the files in it. I've waited well over 10 mins.

When I try to use "ls" cmd in the terminal, it also takes over 10 mins and does not display anything.

When I try to empty the trash, it is EXTREMLY slow. The # of files deleted increases 1 per second (almost). I've waited well over 30 mins and it was still not finished.

How do I diagnose this problem and find a solution?

Update 1: Filesystem is APFS and the directory has more than 12,000,000 small json files. Update 2: Finally counted the # of files and total size of the directory. The directory has 22,203,359 files and 27.59GB.

2
  • Comments are not for extended discussion; this conversation has been moved to chat.
    – Monomeeth
    Aug 7, 2019 at 22:43
  • But the group is talking about the topic to solve the problem?
    – Lano
    Jul 8, 2021 at 10:40

5 Answers 5

3

When I try to use "ls" cmd in the terminal, it also takes over 10 mins and does not display anything.

  1. This is not normal. Something is amiss. I'd see the most likely cause as either a bad filesystem or a failing drive. While surprising, it could be a bad spot on the drive just where the directory is. In the case of a failing drive, macOS will try multiple reads to see if it can retrieve the data.

    /Applications/Utilities/Disk Utility

goto the First Aid tab. Should you find problems, you will have to boot in the recovery manager. command + r at boot time before the beep. click on the utilities tab and select disk utility.

High Sierra's Disk Utility

enter image description here

Yosemite's Disk Utility

enter image description here

  1. Try a safe boot. It's always a good idea to try safe mode when strange things happen.

    Shutdown your machine. Hold down the shift key. Poweron. The boot up will take longer than normal because the filesystem on the startup drive is being checked and repaired as needed. All about safe mode including what features and apps safe boot leaves out. Safe boot uses a software driver instead of using your machines video hardware. http://support.apple.com/kb/ht1455

3
  • On your Disk Utility screenshoot, there’s a red border surrounding a blank area in the list of volumes and disks. Is that supposed to indicate something?
    – EJ Mak
    Aug 7, 2019 at 3:40
  • 1
    Ran the first aid and everything was ok. I finally figured out what's inside the folder. It contains very small json files. I guess it's just normal...there are just too many very small files
    – Moon
    Aug 7, 2019 at 6:28
  • Performance issues with large numbers of small files is absolutely normal. LS does operations on each child (e.g. check if it's a file or directory) - so if a file system lookup takes a thousandth of a second (pretty normal even on an SSD), it can take minutes to list a large enough directory. Jul 8, 2021 at 2:13
3

unix.stackexchange.com once told me that rsync is the fastes way to delete many files:

mkdir empty_dir
rsync -a --delete empty_dir/    yourdirectory/

Maybe this helps here.

2

Deleting a lot of files from Trash via Finder/Empty Trash can be very slow sometimes. It might be easier/faster to run

rm -r ~/.Trash

in Terminal (at least it won't partially block the UI while doing so).

PS: If the files to be deleted are on an external drive you need to delete them from /Volumes/VOLNAME/.Trash instead.

11
  • Still the same. In fact, it does not even delete a single file after 15 mins. The size of directory remains the same unlike emptying trash.
    – Moon
    Aug 7, 2019 at 8:07
  • The was rm works the number of files shouldn’t have a significant impact. Check the disk, make sure you have a recent backup and reboot
    – nohillside
    Aug 7, 2019 at 8:40
  • // I ran first aid and it was fine. Rebooted and tried again. Same result. "rm -r" does nothing for over 10 mins.
    – Moon
    Aug 7, 2019 at 9:50
  • @Moon Does it do nothing (file count doesn‘t change), or does it silently delete files?
    – nohillside
    Aug 7, 2019 at 10:03
  • 1
    That ("directory has at least 10,000,000 json files") would make a worthwhile addition to the question, actually (together with the information whether the filesystem is HFS or APFS). My wild guess here is that the OS can't properly handle directories with so many entries.
    – nohillside
    Aug 7, 2019 at 11:20
1

22 million files at 1 millisecond each will take over 6 hours.

I've no idea if that's a realistic estimate for the unit time, but the Finder is traditionally not the most efficient at file operations.

And I wouldn't trust its 'progress' dialog, either. https://xkcd.com/612/

You should leave the process to run for as long as it takes.

-3

Even "simple" tools like rm are actually several thousand lines of code and full of complicated features that slow them down. In addition, almost all filesystems are not designed to hold large numbers of files in a single directory (10,000 is a sensible upper limit or 1,000 if using a GUI file browser).

Here's a simple script that uses low level UNIX APIs (unlink() and rmdir()) to delete each individual item in the directory, using the nftw() API to scan through the directory.

Simply save it as a text file, make it executable, and run it. I'd expect it to be very fast even with 22 million files.

#!/usr/bin/swift

import Foundation

let path = "/Path/To/Folder"

nftw(path, { child, _, type, _ in
    if (type == FTW_F) { // file
      unlink(child)
    } else if (type == FTW_DP) { // directory "where we have visited children"
      rmdir(child)
    } else if (type == FTW_SL || type == FTW_SLN) { // symlink
      unlink(child)
    } else { // anything else, assume some kind of permission issue or similar
      print("Cannot delete \(String(cString: child!))")
      return 1 // error
    }
    return 0 // no error
  },
  42, // how many file descriptors should we use?
  FTW_PHYS | FTW_DEPTH // do not follow symlinks and do a "depth first" scan (files are provided before their parent dir)
)
3
  • Do you have a reference for the claim about the number of files per directory?
    – nohillside
    Jul 8, 2021 at 6:21
  • I wouldn't be so hard on this poster, assuming the code works. Although, the poster estimates are from the days of DOS. Jul 9, 2021 at 4:04
  • @nohillside There isn't a hard limit, rather performance gradually gets worse the more you have and 10,000 is around when you are likely to run into problems depending what software you are using. As for how I got to that number, it's a combination of real world experience and research over decades spent managing large data sets. It also, obviously, depends on the speed of your storage system. An enterprise grade PCIe SSD is going to handle large numbers better than a Micro SD card. Aug 20, 2021 at 4:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .