How does Apple perform a lossless in-place conversion from HFS+ to APFS?

Question

Since High Sierra, Apple has allowed users to do an in-place conversion of an existing HFS+ file systems to APFS without losing files. This can happen automatically during an OS upgrade or can be done by the user with Disk Utility.

My question is, how, technically, is this accomplished? I'm not asking how a user can do the conversion, I'm asking what actually happens on the disk during the conversion process. I've done some searches but have only found references to the fact that it can be done. This article provides some analogies, and I found some hints on how APFS allows easier resizing of partitions.

Does the software directly convert one file system to another on the fly? If APFS is basically an enhancement to HFS+ I can see this being easy, but my impression is that is not the case.

Does it create two partitions and slowly move content from one to the other, resizing as it goes, or create an image of the old system then restore it to the new system? It seems these would require a fair amount of free space, and could fail if the drive is full.

Something else?

Answer 1

It’s a four step process according to this article from The Ecletic Light Company

1. Read and convert the existing HFS+ metadata (files, directories, etc) to APFS format; writes it to free space on the drive
2. Verifies that the APFS data is correct
3. If no errors, it writes the volume super block in APFS format
4. It deletes the HFS+ meta data leaving just the APFS volume

It converts it a single piece at a time, updating the volume as it goes. This is why you need sufficient free space prior to the conversion process begins.

Answer 2

I don't know much in terms of details, but I'm pretty sure the basic process involves building parallel volume data structures and then switching which one's used.

To understand what that means, you need to know a little bit about how a filesystem stores its files on disk. Somewhat oversimplified, there are two kinds of data being stored: the actual file contents are just stored pretty much as raw data, and then there's some additional database-ish thing that keeps track, for each file, of where that particular file's raw data is stored (as well as "metadata" like file's name, permissions, last modify time, etc).

The raw file contents is pretty much the same between different volumes formats (except for encrypted volumes), but the database-ish metadata will be different between HFS+ vs. APFS vs. FAT vs. whatever.

Most of the disk's contents is (generally) the raw file contents, since the metadata is generally much smaller than the file's actual contents. That means that in converting from one format to another, most of the volume's contents doesn't need to change; only the database-ish thing that stores file locations and metadata needs to be converted. So what you can do is build a new-format database-ish thing in the free space in the volume (copying and converting the contents of the old-format one), and then simply say "ok, use the new one now." (and mark the old one as free space in the new format).

That's the oversimplified view; in practice I'm sure there's a lot of complications and fiddly details that need to be dealt with. But I'm pretty sure this is the basic idea.

BTW, there's a similar trick that's sometimes used (including by Apple) to make multiformat (or "hybrid") disks. For example, Apple's CD/DVD burning and image creation software can make disks that're valid as HFS+ volumes (readable on Macs), and also valid ISO 9660 volumes (readable on a wide variety of other OSes that don't know HFS+ from a hole in the ground). It stores one copy of each file's contents, and two different database-ish things (one ISO format, one HFS+), and sets it up so different OSes will see the one they understand and be able to read the volume.