14

Specifically I'm asking about the mechanism where Finder will show entries that do not exist in the directory on the filesystem as shown by the terminal.

Example 1: /Applications shows App Store.app in Finder, but it's actually loading it from /System/Applications according to Apple.

Example 2: ~/Library/iCloud Drive in Finder actually exists as ~/Library/Mobile Documents on the filesystem.

Example 3: The contents of example 2 are all distorted as Mac does some mapping to resolve to the displayed entries in Finder, e.g. com~apple~Numbers to Numbers and com~apple~CloudDocs/* to *.

Questions:

  • What is causing this? Finder.app? Extended attributes? Some service?
  • Is this link/mapping functionality safe to use as a user for my own uses?
  • Is there documentation somewhere for this functionality?
  • Why is this functionality used, as it distorts 1-to-1 mapping with the filesystem? Even symlinks provide clarity. Is it just enhanced functionality that Apple has added to provide for these unique types of circumstances?
  • Is there any guidance as to when this functionality is better to be used or avoided, such as for convenience, headache, security concerns?

Update:

So firmlinks explains the first example.

The other two (iCloud related) examples seem like they could be caused by Apple's integration of iCloud/FileProvider/(...) APIs. [1] [2] [3] [4] [5] [6] But that seems a bit hardcoded if Finder only attempts to map the iCloud directory tree. I wonder whether other services (DropBox, OneDrive, etc) have any similar directory resolution, which would point to Apple's cloud API implementation being the cause, as 3rd parties are required to use Apple's APIs nowadays from what I understand.

But through Apple's iCloud integration, I wonder whether the extended attributes may be what is triggering Finder to alter its behavior to display contents differently. It seems that com.apple.ubd.prsid and/or com.apple.file-provider-domain-id may be the trigger, as they don't seem to be used anywhere else in the ~/Library/Mobile Documents tree.

> cd ~/Library/Mobile\ Documents
> xattr -r . | grep -E 'com.apple.(ubd)|(icloud\.)|(file-?provider[^#]*$)'
./com~apple~CloudDocs/Desktop: com.apple.file-provider-domain-id
./com~apple~CloudDocs/Desktop: com.apple.fileprovider.detached
./com~apple~CloudDocs/Desktop: com.apple.icloud.desktop
./com~apple~CloudDocs/Documents: com.apple.file-provider-domain-id
./com~apple~CloudDocs/Documents: com.apple.fileprovider.detached
./com~apple~CloudDocs/Documents: com.apple.icloud.desktop
./.Trash: com.apple.fileprovider.trash
.: com.apple.file-provider-domain-id
.: com.apple.ubd.prsid
> xattr -p com.apple.file-provider-domain-id .
com.apple.CloudDocs.iCloudDriveFileProvider/<GUID>
> xattr -p com.apple.ubd.prsid .
<AccountDSID>.CloudDocs

Where <AccountDSID> =

defaults read MobileMeAccounts | plutil -extract 'Accounts.0.AccountDSID' raw -

And <GUID> =

defaults read com.apple.protectedcloudstorage.protectedcloudkeysyncing | \
plutil -extract 'registrySyncIdentifier' raw -

And so it appears the examples I gave have different mechanisms that prompt a similar effect. But are there other instances where this occurs in macOS? Or did I just happen to pick the exact two examples where it exists? I get the impression that may likely be the case.

5
  • 3
    Good question! And two good answers now... which I'd summarize as follows: "You're seeing this stuff because Apple decided to make it available. Beyond that asking questions (who what when where) or for documentation is pointless b/c this is Apple after all."
    – Seamus
    Mar 8 at 6:13
  • 1
    I don't see ~/Library/iCloud Drive in my mac running Ventura
    – mmmmmm
    Mar 8 at 7:57
  • @mmmmmm You don't see ~/Library/iCloud Drive in Terminal, but you do (macOS 14) in Finder! Contents are the same as Finder constructs in the sidebar. I will modify my answer to include this.
    – Gilby
    Mar 8 at 9:17
  • iCloud Drive only started using the File Provider framework with macOS 14. So whilst the other links you have added are good background, it is only link 1 which explicitly investigates the relationship between iCloud Drive and File Provider. The scope of this Q&A is expanding into a discussion about iCloud, iCloud Drive and File Provider.
    – Gilby
    Mar 9 at 0:35
  • Note that many folder names may be translated on computers using a different language. See discussions.apple.com/thread/3014097?sortBy=best for an example and a few details on the topic.
    – jcaron
    Mar 10 at 15:45

3 Answers 3

9

Your question and its examples relate to two distinct features of recent macOS. I will address this by looking at your Example 1 and then Examples 2 and 3.

Example 1:

(Text largely taken from answers by @benwiggy and @Marc Wilson)

For system security and integrity macOS 11 and later are divided into two parts;

  • The Operating System (including bundled apps, fonts, and everything that is installed by Apple) is on a separate, read-only disk volume. (Usually called Macintosh HD).
  • Everything else: your apps, your fonts, documents, etc, is stored on Macintosh HD - Data.

The OS volume is mounted at /System, and the bundled apps like Preview, App Store are in /System/Applications, as you point out. Other applications that you install go to /Applications, but Apple 'merges' the two locations using firmlinks so that applications do not need to be aware of the separation. This separation is hidden from view in the Finder, to keep things simple for the user. You can still see the split in Terminal.

As well as /Applications other top level folders are combined using firmlinks and 1) presented as one by Finder, and 2) combined as one for apps. /Library is perhaps the most important, but also /private and parts of /usr.

As always, Howard Oakley at Eclectic Light is the best resource: https://eclecticlight.co/2023/07/22/how-macos-depends-on-firmlinks/.

Also a link to the WWDC presentation when firmlinks were first introduced: https://developer.apple.com/videos/play/wwdc2019/710/

You ask, can you use this functionality yourself. Whilst you can create your own firmlinks (see near the end of Hoard Oakley's article) you should not do so. The only exception is for apps which, mostly for historical reasons, expect files in their own app specific root level folders.

Examples 2 and 3:

Finder presents 'friendly' views of some of the content of ~/Library/Mobile Documents/. These views are entirely artificial, and are constructed from various elements in the file system. In particular:

  • The folders inside ~/Library/Mobile Documents/com~apple~CloudDocs are displayed inside iCloud Drive. These are mostly folders created by you, but also those of a few apps.

  • The other folders shown by Finder in iCloud Drive have been created (or used by) apps which are on your Mac. In your example ~/Library/Mobile Documents/com~apple~Numbers, the content of its Documents folder is shown in iCloud Drive/Numbers.

  • There are yet more folders in ~/Library/Mobile Documents which Finder ignores and does not present in iCloud Drive. These are folders for apps which are synchronising content between your Apple devices. Some these may have been created in iCloud by apps on your iPhone and others may be for apps which you have deleted years ago!

I should add that what Finder displays in the sidebar as iCloud Drive is the same as it shows (macOS 14) as ~/Library/iCloud Drive. But ~/Library/iCloud Drive is not part of the file system as shown by ls commands in Terminal - it is another Finder only construct.

You should not mess with the content of ~/Library/Mobile Documents. Until recently this was visible in Finder, but now requires Terminal app.

Finder alternatives (e.g. Path Finder) may show ~/Library/Mobile Documents and, in the case of Path Finder, present their own curated view of what is there.

Regarding your explicit questions relating to examples 2 and 3:

  • What is causing this? Finder.app? Extended attributes? Some service? Finder
  • Is this link/mapping functionality safe to use as a user for my own uses? No, you should not mess with it.
  • Is there documentation somewhere for this functionality? Not from Apple - you are not supposed to need to know the underlying functionality. Explore the iCloud and iCloud Drive articles Eclectic Light (Howard Oakley) to get a more in depth understanding.
  • Why is this functionality used, as it distorts 1-to-1 mapping with the filesystem? To make messy things look nice.
  • Is there any guidance as to when this functionality is better to be used or avoided, such as for convenience, headache, security concerns? Look (if you want), don't touch. Use Finder's iCloud Drive.
1
  • But do we know how Finder decides when to customize a directory's contents? What is the trigger? ... I've done some more research and will update my question with new info.
    – MacBeth
    Mar 8 at 20:58
6

The word you're looking for is 'firmlinks'.

As always, Howard Oakley at Eclectic Light is a good resource.

https://eclecticlight.co/2023/07/22/how-macos-depends-on-firmlinks/

Here's a link to the WWDC presentation when they were first introduced.

https://developer.apple.com/videos/play/wwdc2019/710/

4
  • Exactly what I was looking for, thank you. I was wondering whether some other type of mythical link. But does this also apply to all of the iCloud mishmash too? If so, then there would be no easy way to algorithmically determine or query the relationship between vendors and used values other than manual inspection and hardcoding... :/
    – MacBeth
    Mar 8 at 0:13
  • 1
    Firmlinks explain example 1, but not the others.
    – Gilby
    Mar 8 at 1:35
  • Marc, I think this question (which involves 2 distinct macOs mechanisms) is worthy of a single complete answer. May I have your permission to insert your answer into mine?
    – Gilby
    Mar 8 at 9:36
  • @benwiggy agreed with such alacrity that I have jumped the gun and incorporated your two answers into mine. I do hope I am forgiven.
    – Gilby
    Mar 8 at 10:27
1

Example 1: /Applications shows App Store.app in Finder, but it's actually loading it from /System/Applications according to Apple.

The Operating System (including bundled apps, fonts, and everything that is installed by Apple) is on a separate, read-only disk volume. (Usually Macintosh HD. Everything else: your apps, your fonts, documents, etc, is stored on Macintosh HD - Data.

The OS volume is mounted at /System, and the bundled apps like Preview, App Store are in /System/Applications, as you point out. Other applications that you install go to /Applications, but Apple 'merges' the two locations in the Finder, to keep things simple for the user. You can still see the split in Terminal.

Is this link/mapping functionality safe to use as a user for my own uses?

I don't think you can use this functionality for any other use.

3
  • You can create your own firmlinks. Needed, for example, if you have an app which uses a non-standard top level folder. See, the penultimate paragraph in eclecticlight.co/2023/07/22/how-macos-depends-on-firmlinks
    – Gilby
    Mar 8 at 9:05
  • See my comment to @Marc Wilson. Can I combine your answer into mine?
    – Gilby
    Mar 8 at 9:38
  • @Gilby Sure - I just added this as the Applications bit didn't seem to be addressed.
    – benwiggy
    Mar 8 at 9:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .