Backing up a large sparsebundle using Time Machine

Question

I use a large (presently 16GB) sparsebundle as a container for all of my professional work. This is convenient for me because it is an isolated, secure, "portable" entity where all of my work can be stored in one place. It has persisted over three laptops and several hard drive changes (basically copy it from one to another as needed). It has worked great. In the past, I've used ChronoSync as my backup utility of choice: separate documents to backup drive excluding sparsebundle, then a second document to backup the mounted bundle as a filesystem -- thus, entire bundle is not copied every time, just changed files. But, I have recently started using Time Machine given the wonderful integration it provides. However, I'm not convinced it is handling this large sparsebundle properly:

1. I can't seem to access it via the Time Machine itself. The only option presented is "restore". I don't want to "try" that out. I created a test bundled (100MB) with the same security. I put a few files in... manual backup via TM... added a few more... manual backup... deleted a few... manual backup. However, when I restore those via the TM interface, the changes don't seem to track. There is indeed a backup instance for each manual backup, but the instances don't correspond to the changes made. Perhaps I am going faster than the system is actually writing to disk? (probably 1 minute between changes).

2. Directly accessing the filesystem (Backups.backupdb) shows the large 16GB sparse bundle present in most (all?... haven't checked) backup instances (e.g., 2013-02-28-042451). The filesystem reports it as 16GB, prompts for the password if I attempt to mount it, but then fails with "no mountable file system". I guess this is to be expected as I don't think Time Machine backups up the ENTIRE bundle each time... but I can't even mount the ORIGINAL bundle from the first backup, which I would presume is a true copy.

The questions are: does Time Machine definitely handle large sparsebundles properly? Should I be concerned about my testing with the small bundle? Is there any way to access individual files within a Time Machine backup of a large sparsebundle, or does the whole bundle need to be restored?

Answer 1

Time Machine works at a file level, with no facility to perform incremental changes within files. As such, your Sparsebundle may be backed up in it's entirety every time it changes in the slightest, depending on how large it is. Of course, you have to wait up to 1 hour (+the time it take to make the backup, depending on how large the queue is and where your file sits in it) to ensure those changes are included in the backup. Also if your Sparsebundle is in use (mounted...) then it may well skip it until the file lock is freed

This is a terrible system, and one that we might not see change until the underlying filesystem is suitable upgraded (or replaced) to include such useful facilities as incremental block level changes rather than simple file level ones, and/or deduplication etc. One early victim of this scenario were users who used the original Filevault system for encrypting their home folders. Time Machine would not backup their home folders until such time as they logged out because the Sparseimage file was constantly locked by the fact the user had it mounted. And even when the user did log out, it would proceed to make a hugely inefficient backup of the whole thing again and again - on the assumption that they simply logged out and didn't just turn off etc... Not very clever. To try to ameliorate this the Sparseimage spec was ammended to allow for Sparsebundles. Instead of a single big file, a sparse bundle is a bundle (directory) containing a number of files called bands, each in the order of 8 MB in size. This means even though to the end user the sparse bundle appears as a single file, it is composed of smaller files. As of Mac OS X 10.8, the bands are 8.4 MB each. When the content of the image changes, one or more band files is changed, created, or deleted. This allows backup software (such as Time Machine) to operate more efficiently, but it's just a bodge to attempt to mimic block level changes in individual files, which is limited to 8Mb "blocks"...

So to answer your questions directly, 1) it handles them properly, where properly means the same as any other file, it's just that your particular use (leaving it open and mounted) may not result in efficient backups that are taken regularly, especially if you rarely unmount the file, and 2) yes, you will need to pull back the whole file to view it's contents. The TM restore interface is also file specific. It may have quick-look plugins to allow you to view simple files inline like JPGs etc, but not for a complex file like a sparsebundle.

On the bright side, you already have a licensed copy of ChronoSync, which is super useful, and I would continue to use this to perform incremental backups of your sparsebundle whilst it is mounted, you can use the same drive as your TM images too.

Answer 2

The only time I've seen the error you report was when I stored a sparse disk bunds inside another sparse disk bundle and both were remotely mounted on a fairly slow network storage device such as a Time Capsule.

In those cases, I've often excluded the original image directory from the Time Machine backups, since it works in theory (and most of the time in practice), you can't get any benefits of browsing those files on a Time Machine backup.

I usually set up rsync or another incremental backup tool to copy those files on a schedule I like (perhaps automating it all with Lingon 3).

By restoring the file to a local drive, you'll know if the failure to mount is due to an actual corruption or just network indirection and time out.

Once it's copied to an attached or internal disk, you might also be able to use Disk Utility to patch up filesystem errors although with encrypted sparse disk bundles, that is often not successful.