Selectively copying code and config files to iCloud

Question

It's common knowledge that it's a bad idea to symlink (or straight up store) your code (with .git repos) and various config files (plist, ~/.config, zsh_history) in iCloud. It's gonna be a mess when you inevitably start seeing myfile (1), myfile (2) piling up in various subfolders. I tried, and indeed that's what happened.

I tried to go a different route: made a launchd-based daemon that runs my script hourly, in which I use rsync to copy files from where they are to iCloud-synced directory. This is better, and worked for a bit, but it's always breaking for some reason. One day rsync stops having permissions at random, another day my daemon just stops running. And somehow, the myfile (1), myfile (2) problem crops up anyway (albeit rarely).

So my question is — is there a way (a tool) that reliably and efficiently copies a few specific files and folders into an iCloud-synced dir? Without causing a mess, and without randomly seizing operation. Any help appreciated.

Answer 1

For those who will find this by googling, here's an answer that gets you as close to working reliably as I could.

TLDR: Use cron with a script that runs cp/rsync commands. Give /usr/sbin/cron Full Disk Access in your Security & Privacy settings. This is more stable than launchd.

The long story

Technically, copying files into iCloud periodically should work without issues, especially if there's only one source. It didn't for me.

There are 2 "native" ways to implement periodic tasks in MacOS.

1. Cron (supposed to be deprecated)
2. Launchd (build your own daemon)

I went with #2, because of cron deprecation. It ended up being quite a hassle to wrap my script into a .app package to give it "Full Disk Access", but it sort of worked (thanks to the linked info). I also setup 2 log files (stdout/stderr) to make sure it's working.

I started having problems:

1. The log files started being filled with permission issues. I solved some of them by disabling "checksum" checks in rsync. Still, issues continued, even in simple single file cp calls. I checked everything and couldn't figure out why issues persisted.
2. Occasionally I started getting myfile (1) myfile (2) files show up due to conflicts, which shouldn't really be there. I haven't opened or changed these files on other devices.
3. The job would just stop working at random. I couldn't figure out why. I would see an old last entry in logs.

Finally, after posting here, I gave up and tried to switch to cron (without much hope, because I didn't see how changing to a different runner would fundamentally address any of these issues). I added /usr/sbin/cron to Full Disk Access, and setup my hourly script like this:

0 * * * * /Users/max/Documents/Configurations/Executables/hourly-job >/Users/max/Library/Logs/HourlyJob/HourlyJob-stdout.log 2>/Users/max/Library/Logs/HourlyJob/HourlyJob-stderr.log

(These log locations allow me to see them in Console.app.)

I removed all the launchd stuff.

Surprise, now it just works. No more permission issues. So far no more conflicts, but that's harder to check so soon. They might recur later.

Also interestingly, according to this, cron is unlikely to ever be removed. It's been 9 years since that answer and cron is still here.

For those interested, here's how I copy files in my script:

#!/usr/bin/env bash
set -e

echo "job @ `date +"%F %T %Z%z"`"

HOME="/Users/max"
LIBRARY=$HOME/Library
APP_SUPPORT="$LIBRARY/Application Support"
ICLOUD="$LIBRARY/Mobile Documents/com~apple~CloudDocs"
LOGS="$LIBRARY/Logs"
CLOUD_CONFIGS="$HOME/Documents/Configurations"
PREFERENCES="$LIBRARY/Preferences"

function rsync_file() {
  rsync -pgohuWE --inplace --stats "$1" "$2"
}

function rsync_dir() {
  rsync -auhOE --delete --stats "$1" "$2"
}

echo "syncing ~/Engineering"
rsync_dir "$HOME/Engineering" "$ICLOUD/"

echo "copying ScanSnap profiles"
cp -fp "$PREFERENCES/jp.co.pfu.ScanSnap.V10L10.plist" \
       "$CLOUD_CONFIGS/ScanSnap/jp.co.pfu.ScanSnap.V10L10.plist"

echo "syncing Bartender preferences"
rsync_file "$PREFERENCES/com.surteesstudios.Bartender.plist" \
           "$CLOUD_CONFIGS/Bartender/"

echo "copying zsh_history"
cp -fp "$HOME/.zsh_history" "$CLOUD_CONFIGS/Zsh/zsh_history"

# … etc …