1

Is the time required for incremental Time Machine backups greatly affected by the number of files being backed up? I have a large number of files on my Mac that very rarely change and aren't "mission critical". If they greatly affect backup time I'll exclude them from Time Machine but if they don't I'll just backup everything. Backup size is a non-issue for me.

If it matters, I have Time Machine backing up to a remote box that is usually on the local network.

1

Incremental backups only backup what has changed. The number of unchanged files is irrelevant during an incremental backup. If you suddenly change all of the "large number of files", the next incremental backup will take significantly more time.

4
  • I need a bit more evidence. One could imagine a worse case but reasonable implementation where Time Machine hashes every file that may need to be backed up. In that case the number (and size) of files certainly could slow down back ups even when nothing changes. – Praxeolitic Oct 1 '16 at 5:00
  • I have 243 million files on my hard drive, according to Disk Utility. Looking at Console for the Time Machine process backupd, it tells me that at 6:11:23 PM last evening it started an incremental backup process. At 6:11:28 it had determined 529 files needed to be backed up. Granted, there are some excluded files so the 243 million number is high, but even if it's actually only half that, the process you describe isn't possible in 5 seconds for 121.5 million files. Looking back through the logs, the 4-5 second timeframe is pretty consistent. – tubedogg Oct 1 '16 at 5:08
  • That methodology seems fair but situation specific so I should test it for myself. How do I determine the start and stop time for a backupd process like you did? Also, if you know, it would be good to add to the answer how Time Machine identifies modified files so fast. Is it mtime? – Praxeolitic Oct 1 '16 at 6:38
  • I’ve added an answer rather than rewrite this one. Depending on the scan type, the backups can take a bit longer and there are now excellent tools and logs to measure this precisely. – bmike Dec 27 '20 at 13:47
0

There are four types of backup, each has different time constraints that you can time down to the fraction of a second to convince yourself of the speed of each.

  • First backup (count of changed files is the same as all files by definition)
  • Fast scan using file system events (count of files changes has little effect other than the time to back each up)
  • Deep scan using filesystem traversal on both the source and destination (count of files changed has no effect, all files are counted and compared)
  • Problem backup where you have to do a consistency check on the destination or purge files before the backup starts ( skip these as they are edge cases)

To time things and simulate a deep scan:

time tmutil compare 

To time a full backup

time tmutil startbackup —-block

I find backups to sparsebundles the worst to mange since they are finicky and you’re adding several layers of error with network and embedded filesystem with sparse storage overhead. Direct attached backups are superior for me and I use arq or rsync if I must have network backups. You can rsync or arq from a local time machine destination which gets the best of both worlds.

If you prefer a graphical tool to opening the command line, go immediately to this web site to get the time machine mechanic. It collects timing data from the system logs and does all sorts of excellent checking on backup health and statistics in general, far past simple timing measurements.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .