1

I wish to use rsync to copy a lot of data into a OneDrive-synced directory regularly, and leave it up to OneDrive itself to sync that into the cloud. rsync is my tool of choice as I am using it for various other activities already - but I'm not totally set on it if there is a better way.

We're talking low GB ranges here, but very few of those files change regularly (a few dozen MB per day maybe).

Unfortunately, even though I have tried to tune the rsync command as much as possible, it looks like on each rsync run a lot is detected as changed files by OneDrive, and OneDrive runs a very long time (hours) while producing high CPU load as well.

My command looks like this:

rsync -r --no-t -u -W --delete --stats -h $HOME/Data/ $HOME/OneDrive/Backup

I guess a big part of the problem is that OneDrive uses filesystem "magic" to support files that are not actually downloaded right now, and it's a little hard to rsync to work with that. Basically it would be fine for me if only files that are newer in the source are copied (without looking at the content at all, and giving up early if OneDrive itself modifies any of the times). The -W is to avoid downloading all the files from the OneDrive cloud everytime the rsync runs for its content-based delta-update (assuming that happens at all when copying locally).

Does anyone know which options to use with rsync to make this as efficient as possible regarding the OneDrive sync? In other words, I wish for OneDrive to detect as few changes as humanly possible (less upload activity), and not to require rsync to actually open the files to check their content (less download activity).

Versions:

macOS Monterey 12.5.1
OneDrive 22.191.0911.0001
3
  • Excellent update, you might have to switch to using source control (git or similar) rather than OneDrive for this. You’re using the new file provider that can only optimize if you leave the files permanently in OneDrive rather than back them up periodically… this is precisely why Apple implemented sparse disk bundles for their previous Time Machine implementation.
    – bmike
    Sep 26, 2022 at 11:35
  • 1
    Oh man... thanks for that, @bmike (and feel free to create an answer from it). I guess I'll switch everything around and just do daily .tar.gz's with the changed files - better than nothing, and one smallish file per day should be something OneDrive can handle. ;)
    – AnoE
    Sep 26, 2022 at 11:51
  • Your solve is likely the best. Hide them in a hidden folder FTW
    – bmike
    Sep 26, 2022 at 12:24

0

You must log in to answer this question.

Browse other questions tagged .