Take the 2-minute tour ×
Ask Different is a question and answer site for power users of Apple hardware and software. It's 100% free, no registration required.

Given a hard disk with an encrypted Core Storage volume (but not the decryption password, because the backup service should not have access to that), how would one go about backing it up in a way that allows for pushing it to a cloud storage provider (like Amazon S3) and for incremental backups in the future (because you don't want to to push a full 1TB every day when only a couple of blocks have changed)?

share|improve this question

1 Answer 1

Proposed solution:

You have an Amazon EC2 instance, with elastic block store large enough to hold the whole of the image you intend to backup:

backup-host.yourdomain.com:
/mnt/EBS/my-desktop-backup/coreimage.dmg
/mnt/EBS/my-laptop-backup/coreimage.dmg

Where:

/dev/ebs-disk-001 -> /mnt/EBS/my-desktop-backup
/dev/ebs-disk-002 -> /mnt/EBS/my-laptop-backup
etc.
etc.

or

backup-host.yourdomain.com:
/mnt/EBS/my-desktop-backup_coreimage.dmg
/mnt/EBS/my-laptop-backup_coreimage.dmg

Where:

/dev/ebs-disk-001 -> /mnt/EBS

Your initial backup would take a long while to sync, but if you employ rsync to sync, then you can eventually have the remote image catch up to your local image's changes.

Once it is caught up, you can then initiate an EBS snapshot on Amazon's side for the EBS volume containing your encrypted image.

Rinse and repeat for each backup period + snapshot you want to have backed up to the remote server, taking the following items/requirements into account:

  • The Encrypted Image needs to be unmounted.
  • The remote image copy needs to be 100% synced with the unmounted local image.
  • The snapshot needs to be done with the remote EBS volume sync'd, filesystem buffers flushed, and with no changes pending.

With this, you will be able to do the incremental backups using Amazon's cloud technology.

S3 has some serious limitations, which would not suit your needs, for this particular purpose.

The EC2 instance, if fully backed by EBS, can be shut down when you are not doing a remote sync. Ie, when your backup kicks off, you can have it fire up the instance via Amazon's EC2 API, and get the dynamic name or IP address. Once it confirms it is up, it can kick off the rsync backup. When done, it can shutdown the remote image and initiate an Amazon EBS volume snapshot action.

Edit:

rsync does chunk/block level diffs for larger files. You can specify the size of the block diff:

--block-size=SIZE

You can also specify the data stream being sent to the remote server to be compressed, saving you on traffic.

Caveats about S3 vs EBS:

Unless the solution you employ supports splitting the single large file into segments and sending them in parallel, Amazon S3 throttles the data being sent down to under 400KB/sec after a certain size.

I employ rsync differential backups on my servers to S3 as compressed tarballs. Even at tarball sizes of about 500MB, S3 will throttle. In order to work around this, you need to split the file you are sending up into parts, otherwise, the backup to S3 will take forever.

Whereas an EC2 instance with EBS volumes will be faster and not require the need to split files, simplifying backup and restoration.

share|improve this answer
1  
So rsync does chunked diffs and uploads only the parts that have changed? –  Thilo Apr 24 '13 at 4:03
    
@Thilo Yes, that's one of the key benefits of rsync. And you might want to check out Arq or any of the other backup solutions for S3. –  patrix Apr 24 '13 at 5:46
    
@patrix: Yes, I am quite interested in Arq. But I could not find documentation on how it handles large files that only change a bit every time. –  Thilo Apr 24 '13 at 6:07
    
@Thilo It's in the second paragraph beneath "Wayback Machine" on haystacksoftware.com/arq: "Only the changes are uploaded for each backup, minimizing network and storage usage". But I agree that changes could mean file-level as well. Maybe ask their support directly? And let's stop the discussion here now before a moderator jumps in :-) Ping me in chat if needed. –  patrix Apr 24 '13 at 6:17
    
As noted in my answer, I would recommend against S3, having tried using it as a backup backend, there are multiple negatives to employing it, especially if you intend to do rsync/block diffs. I'll elaborate in the answer. The ARQ software looks interesting. But I thought the question was how to backup an encrypted filesystem in it's encrypted form and not individual files. –  Wing Tang Wong Apr 24 '13 at 16:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.