Here's the goal: I have a backup server with a ZFS filesystem with built in compression and snapshots. All our machines backup there nightly with rsync, and then a ZFS snapshot is taken, so we have backups of how eveach machine looked at the end of every day.

The problem is that with Mac OS X, the resource forks are not copied since the remote machine does not support resource forks. What I would like to do is rsync to a .DMG file which resides on the backup server. This way, I'd have an exact, bootable image of how my system looked each night.

As a bonus, the backup server is also a NetBoot server, so this would actually allow me to boot from how my mac was at any point in history.

I created an image of my Mac's drive and copied it to the backup server. Now, when I open it on my mac and run rsync, it runs for a period (sometimes more than an hour, sometimes less) and then gives me a ton of I/O errors. The image is stored on a RAID array, I am sure there are no I/O errors.

I think the connection with the Samba share may be getting overwhelmed... I've tried with AFP also, and I get the same result... What could be causing this and how can I resolve it? Or, any other ideas on how I can update the remote DMG file?

  • 3
    Rather than re-invent TimeMachine using open source/unixy tools (which I adore and love) - have you considered running the free CrashPlan software on both machines? They are giving away software that is very advanced and well supported if you need help from them. – bmike Jun 6 '11 at 15:57
  • Hmmm, I'll have to look into that! The reason I'm "re-inveting" TimeMachine, FYI, is that TimeMachine requires a large USB drive for each machine, and in our setup we don't have that, rather, we have a massive onsite backup RAID. – Josh Jun 6 '11 at 16:02
  • I totally get your reasons. ZFS is great and reusing existing infrastructure makes so much sense. For several clients with 10 to 30 macs - the sweet spot has been a mac mini server sharing one drive to all mac clients for Time Machine. That server has crash plan to the main server for tape/offsite rotations (and the sparsebundles encapsulate HFS and metadata). DeployStudio handles the bootable images and net restores. Good Luck and let us know how it goes. – bmike Jun 6 '11 at 16:25
  • Another +1 to @bmike's CrashPlan suggestion. Uses Hadoop under the hood and can do incrementals like TimeMachine. Free to push backups to another drive or computer. UI isn't as nice as TimeMachine though. – Ian C. Jun 6 '11 at 17:43
  • +1 for crashplan. It's a great piece of free software and has the added benefit that you can easily add more points of backup - including their own servers for a fee. – cregox Jun 7 '11 at 13:49

Setup a new volume on your Nexenta/OS box for your backup and set it to share out as an iSCSI LUN. Then use one of several iSCSI initiators for MacOS X to attach the backup volume on your back. On your mac, format the volume as HFS+. Now you can use CCC or rsync, etc. to do your backups.

  • 2
    This is a fantastic idea. I'll try that after I finish work for the day! – Josh Jun 6 '11 at 16:03
  • 1
    +1 - I was thinking of a way to use iSCSI to get you there and encapsulate the mac specific data. The less steps, the better. – bmike Jun 6 '11 at 17:00
  • Bah! Just as I was typing that this is working great, the connection was lost. But I still have hope for this solution... – Josh Jun 7 '11 at 12:09
  • @Josh: Do you know why the connection was lost (physical unplug, etc). Also do you have a separate NIC for your iSCSI connection? – tegbains Jun 7 '11 at 19:26
  • I don't know what's causing the connection to be lost, but since it happens with all protocols now, AFP, CIFS, NFS and iSCSI, it's a problem with the server. I think the data transfer is just overwhelming the virtual machine and it becomes unresponsive... but I'll ask another question, probably on the unix&linux stack exchange. – Josh Jun 8 '11 at 12:23

Have you looked into using Carbon Copy Cloner? I use this to a manual bit-for-bit backup of my SSD to a network drive once a month or so and it works flawlessly. I know it does incremental backups, and I've had no trouble dumping my drive over gigabit ethernet.

  • Unfortunately, when I use CCC for an incremental backup, it just runs rsync and I get the same result :-( – Josh Jun 6 '11 at 15:41
  • When you dump your drive to the network drive, what hosts the network drive? OS X? Linux? How do you connect? Samba? AFP? Do you use CCC direct to the network drive, or, do you go to a DMG on the network drive? – Josh Jun 6 '11 at 15:44
  • @Josh: You could CCC to the mounted Disk Image – tegbains Jun 6 '11 at 15:57
  • That's what I was doing -- and CCC was running rsync and I was getting the same I/O errors :-( – Josh Jun 6 '11 at 15:59

Not the exact answer to your question, but you can rsync resource forks too. Here are really nice articles about:

ps: do you use solaris or freebsd?

So, what im tried now:

I have a remote OS (not mac) and mouned it via fuse/sshfs (over the slow ADSL).

sshfs user@example.com:/path/to/dir /Users/me/Mount/myfs -oauto_cache,reconnect,volname=MyFs

On my macbook entered this:

/usr/bin/rsync -avE /Users/me/tmp/bk /Users/me/Mount/myfs/tmp/test

(the /tmp/test already exists on the remote computer)

and got this:

building file list ... done

sent 311127 bytes  received 186 bytes  15186.00 bytes/sec
total size is 6874  speedup is 0.02

As you can see, resource forks are converted into dot_underscore files and copied into remote location. (for example, the custom icon for the directory bk too).

Therefore i don't understand, why you can't rsync into remote zfs?

  • 1
    No matter what I have tried, I can't get rsync to copy the resource forks, I think because the ZFS system doesn't support them... I'm not sure. I first tried with a Debian kBSD server, but have switched to Nexenta (OpenSolaris kernel + Debian core) because it has NFS and Netatalk support. – Josh Jun 6 '11 at 15:43
  • And the 3rd link? (that show a OS/FS independendent solution) – jm666 Jun 6 '11 at 15:57
  • 1
    i'm really interested in this. Here is another link for zfs + timemachine. blogs.oracle.com/constantin/entry/zfs_and_mac_os_x – jm666 Jun 6 '11 at 16:18
  • Ah, I see, you're mounting the directory and rsync ing locally! I was trying to rsync from the Mac to the server! I will try this and get back to you. – Josh Jun 6 '11 at 16:50

I don't know why, but the connection to the server was being broken during the file copy and I am assuming it was because the high amount of data being transferred at such high speed was more than the backup server could handle; it would become unresponsive for a short period and the Mac would forcibly disconnect the DMG, and the backup would fail.

I found a solution: before backing up, in System Preferences, I lowered my Ethernet NIC's speed from 1000Mbps to 100Mbps. Then, the backup seemed to work flawlessly, because the data rate was constrained and the backup server didn't get overwhelmed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .