Invalid fsroot tree, what it really means and file integrity?

Question

Mac Pro model 5,1 (mid 2012, 3.33 Ghz 6-core Xeon) with 32 GB of stock RAM and the stock video card. I am running High Sierra fully updated.

I have a fairly new 4 TB Seagate HDD (3.5") that I use strictly for data; non-booting, no OS. I use it for my 1.5 TB Home folder since it's way too big for my SSD boot drive. When I installed it late last year, I formatted it as APFS and used CCC (Carbon Copy Cloner) to clone my old drive over to it.

It's been working fine without issues as far as I know. While I was in Disk Utility I ran Disk First Aid on this hard drive and it found the following:

Checking the APFS volume superblock.
Checking the object map.
Checking the fsroot tree. 
error: inode_val: object (oid 0x3): invalid nchildren (-1) 
fsroot tree is invalid. 
The volume /dev/rdisk3s1 could not be verified completely. 
File system check exit code is 0. 
Restoring the original state found as unmounted. 
Operation successful.

Disk Utility gives you the green checkmark "complete", and you have no idea there is an issue unless you click "see details". According to the documentation for fsck_apfs, exit code 0 means no errors.

I've already looked into third-party tools and there are none that can repair APFS drives. I've also attempted fsck via the terminal but that results in the exact same error and messages. There seems to be no way to fix an invalid fsroot except for reformatting the drive.

I bought another 4 TB drive, formatted it as HFS, and used CCC to clone the "bad" APFS drive over and get the machine back into use. The new HFS drive passes Disk First Aid without errors.

• What does the "invalid nchildren" error mean?

• How do I know CCC copied all my files without corruption? Since it was apparently operating without issues except for the First Aid error, I have no idea if any files are missing or corrupted.

• Is there another way to fix this drive? No 3rd party utilities can repair, but perhaps through debug mode finding the files that caused the error and restoring or deleting those?

EDIT:

Running fsck_apfs again in debug mode -d. Disk was unmounted and it's not the boot disk, yet still had to use sudo to overcome permission denied errors.

Mac-Pro:~ ••••••••$ sudo fsck_apfs -d /dev/disk3s1
Password: ••••••••
** Checking volume.
** Checking the container superblock.
** Checking the space manager.
** Checking the object map.
** Checking the APFS volume superblock.
** Checking the object map.
** Checking the fsroot tree.
error: inode_val: object (oid 0x3): invalid nchildren (-1)
obj-id:        3 type: Inode      
private-id: 3 parent-id: 1 cr/mtime: 1638063393117254350/1663519361466993480 
gen-count: 32501214 nchildren: -1 
def-prot-class: 0 
uid/gid/mode: 0/0/0x41a4 bsd_flags: 0x0 internal_flags: 0x8000 name: NO-NAME
   fsroot tree is invalid.
** The volume /dev/disk3s1 could not be verified completely.

FWIW: I have the original 4 TB APFS drive with the invalid fsroot tree in my hands untouched if I need to do further troubleshooting and/or recovery.

Answer 1

The meaning of the file system check error message below:

error: inode_val: object (oid 0x3): invalid nchildren (-1)

is the following:

inode_val

Each object (file, directory, link, whatever) on your file system is represented as a data structure known as an "inode". The inode contains metadata about the object, such as for example when it was created, when it was last modified, how large it is, who owns it, etc.

An "inode_val" is simply the value (contents) of the inode.

oid

Each object has a unique number for identification purposes. This is also otherwise referred to as the "inode number". In this case the inode number is 3.

This is important for you as the number is very low. This usually means that this was one of the very first files or directories created on the file system.

The first inode numbers are assigned as follows:

  0 = invalid (can't happen)
  1 = the parent of the "/" (root) directory - doesn't really exist
  2 = the root of the file system ("/")
  3 = the private directory ("private-dir")
  6 = directory for snapshot metadata
  7 = holder for purgeable files (not an actual directory)

On a factory default setup, the root of the file system ("/") has object id 2.

In your case the object has inode number 3. This means that the inode directory represents the special "private-dir" directory. This is not to be mistaken with "/private" or anything like that.

The "private-dir" directory is not visible from user space - i.e. programs that you run on your computer cannot see this directory.

It is used by file system implementations to hold data that cannot otherwise be represented in the standard file system structure, but needs to be stored on disk. It is not used by Apple.

In essence, if for example a Linux implementation of APFS was made, and they needed to store some extra special information about files and directories that is used on Linux, but not on macOS, they could store it here. That way it is stored safely on disk, and Apple's software will know not to mess with it.

nchildren

Inodes that represent directories have a field name "nchildren". Other types of inodes do not have this field, instead they have a field name "nlinks". So basically, the existence of the "nchildren" means that this particular inode is a directory.

The meaning of "nchildren" is to record the number of entries in the directory. I.e. if nchildren had the value 8, it would mean there there are 8 files or sub-directories in that directory.

In your case the value is -1. This is not valid number, as it does not make sense to have negative 1 files or folders in a directory. The operating system will simply regard that value as invalid and treat it the same as if it had the value 0.

To sum this up, there's absolutely nothing practically wrong with your file system. It is just a file system that wasn't created by Apple's standard tool for creating file systems.

There's probably a small bug in CCC that made it create the inode with nchildren listed as -1 instead of 0. Apple's fsck tool notes this as an error, but otherwise states that the file system is good. There's no problem in practice.

You haven't lost any files or folders, there's not going to be any performance or stability issues in the future. All is good.

Answer 2

Backup up files (or using file recovery software) then erasing the drive works in tricky cases where a self diagnosis or self rescue exceed the time you can spend on errors in hardware or file system integrity.

With APFS and snapshots, each single snapshot can double the time needed to check and eventually, you may just need to have backups (on more than one disk) and be willing to move on and focus on integrity of files, not expecting bomb-proof filesystems and an fsck that can overcome all errors without data loss.

• https://eclecticlight.co/2021/03/25/time-machine-to-apfs-maintenance-and-repair/
• https://eclecticlight.co/2022/05/22/last-week-on-my-mac-will-disk-utility-ever-work-properly/

Much of the above blog is in perfect alignment with my and several peer’s perspective and while we delight in some of the advances Apple has made, there are some things we’ve certainly lost as complexity marches on in computing systems. Sometimes, you’re going to have to erase a disk to know if it’s faulty or the accounting got messed up in an unrecoverable manner.

Answer 3

Here's a screenshot from a high level user on the Apple forum:

Basically since the exit code is 0 (presumably the reason for the green checkmark and "operation successful"), ignore the error... nothing to see here, should have never clicked "see details", and move along.

It's all just so disappointing and sloppy.

---

As far as using APFS on conventional spinning drives... don't. That seems to be the consensus.