11

Here's the goal: I have a backup server with a ZFS filesystem with built in compression and snapshots. All our machines backup there nightly with rsync, and then a ZFS snapshot is taken, so we have backups of how eveach machine looked at the end of every day.

The problem is that with Mac OS X, the resource forks are not copied since the remote machine does not support resource forks. What I would like to do is rsync to a .DMG file which resides on the backup server. This way, I'd have an exact, bootable image of how my system looked each night.

As a bonus, the backup server is also a NetBoot server, so this would actually allow me to boot from how my mac was at any point in history.

I created an image of my Mac's drive and copied it to the backup server. Now, when I open it on my mac and run rsync, it runs for a period (sometimes more than an hour, sometimes less) and then gives me a ton of I/O errors. The image is stored on a RAID array, I am sure there are no I/O errors.

I think the connection with the Samba share may be getting overwhelmed... I've tried with AFP also, and I get the same result... What could be causing this and how can I resolve it? Or, any other ideas on how I can update the remote DMG file?

5
  • 3
    Rather than re-invent TimeMachine using open source/unixy tools (which I adore and love) - have you considered running the free CrashPlan software on both machines? They are giving away software that is very advanced and well supported if you need help from them.
    – bmike
    Jun 6, 2011 at 15:57
  • Hmmm, I'll have to look into that! The reason I'm "re-inveting" TimeMachine, FYI, is that TimeMachine requires a large USB drive for each machine, and in our setup we don't have that, rather, we have a massive onsite backup RAID.
    – Josh
    Jun 6, 2011 at 16:02
  • I totally get your reasons. ZFS is great and reusing existing infrastructure makes so much sense. For several clients with 10 to 30 macs - the sweet spot has been a mac mini server sharing one drive to all mac clients for Time Machine. That server has crash plan to the main server for tape/offsite rotations (and the sparsebundles encapsulate HFS and metadata). DeployStudio handles the bootable images and net restores. Good Luck and let us know how it goes.
    – bmike
    Jun 6, 2011 at 16:25
  • Another +1 to @bmike's CrashPlan suggestion. Uses Hadoop under the hood and can do incrementals like TimeMachine. Free to push backups to another drive or computer. UI isn't as nice as TimeMachine though.
    – Ian C.
    Jun 6, 2011 at 17:43
  • +1 for crashplan. It's a great piece of free software and has the added benefit that you can easily add more points of backup - including their own servers for a fee.
    – cregox
    Jun 7, 2011 at 13:49

4 Answers 4

5

Setup a new volume on your Nexenta/OS box for your backup and set it to share out as an iSCSI LUN. Then use one of several iSCSI initiators for MacOS X to attach the backup volume on your back. On your mac, format the volume as HFS+. Now you can use CCC or rsync, etc. to do your backups.

7
  • 2
    This is a fantastic idea. I'll try that after I finish work for the day!
    – Josh
    Jun 6, 2011 at 16:03
  • 1
    +1 - I was thinking of a way to use iSCSI to get you there and encapsulate the mac specific data. The less steps, the better.
    – bmike
    Jun 6, 2011 at 17:00
  • Bah! Just as I was typing that this is working great, the connection was lost. But I still have hope for this solution...
    – Josh
    Jun 7, 2011 at 12:09
  • @Josh: Do you know why the connection was lost (physical unplug, etc). Also do you have a separate NIC for your iSCSI connection?
    – tegbains
    Jun 7, 2011 at 19:26
  • I don't know what's causing the connection to be lost, but since it happens with all protocols now, AFP, CIFS, NFS and iSCSI, it's a problem with the server. I think the data transfer is just overwhelming the virtual machine and it becomes unresponsive... but I'll ask another question, probably on the unix&linux stack exchange.
    – Josh
    Jun 8, 2011 at 12:23
3

Have you looked into using Carbon Copy Cloner? I use this to a manual bit-for-bit backup of my SSD to a network drive once a month or so and it works flawlessly. I know it does incremental backups, and I've had no trouble dumping my drive over gigabit ethernet.

4
  • Unfortunately, when I use CCC for an incremental backup, it just runs rsync and I get the same result :-(
    – Josh
    Jun 6, 2011 at 15:41
  • When you dump your drive to the network drive, what hosts the network drive? OS X? Linux? How do you connect? Samba? AFP? Do you use CCC direct to the network drive, or, do you go to a DMG on the network drive?
    – Josh
    Jun 6, 2011 at 15:44
  • @Josh: You could CCC to the mounted Disk Image
    – tegbains
    Jun 6, 2011 at 15:57
  • That's what I was doing -- and CCC was running rsync and I was getting the same I/O errors :-(
    – Josh
    Jun 6, 2011 at 15:59
2

Not the exact answer to your question, but you can rsync resource forks too. Here are really nice articles about:

ps: do you use solaris or freebsd?

So, what im tried now:

I have a remote OS (not mac) and mouned it via fuse/sshfs (over the slow ADSL).

sshfs user@example.com:/path/to/dir /Users/me/Mount/myfs -oauto_cache,reconnect,volname=MyFs

On my macbook entered this:

/usr/bin/rsync -avE /Users/me/tmp/bk /Users/me/Mount/myfs/tmp/test

(the /tmp/test already exists on the remote computer)

and got this:

building file list ... done
bk/
._bk
bk/.DS_Store
bk/._.DS_Store
bk/Icon\#015
bk/._Icon\#015
bk/test.rtfd/
bk/test.rtfd/.LSOverride
bk/test.rtfd/TXT.rtf

sent 311127 bytes  received 186 bytes  15186.00 bytes/sec
total size is 6874  speedup is 0.02

As you can see, resource forks are converted into dot_underscore files and copied into remote location. (for example, the custom icon for the directory bk too).

Therefore i don't understand, why you can't rsync into remote zfs?

4
  • 1
    No matter what I have tried, I can't get rsync to copy the resource forks, I think because the ZFS system doesn't support them... I'm not sure. I first tried with a Debian kBSD server, but have switched to Nexenta (OpenSolaris kernel + Debian core) because it has NFS and Netatalk support.
    – Josh
    Jun 6, 2011 at 15:43
  • And the 3rd link? (that show a OS/FS independendent solution)
    – clt60
    Jun 6, 2011 at 15:57
  • 1
    i'm really interested in this. Here is another link for zfs + timemachine. blogs.oracle.com/constantin/entry/zfs_and_mac_os_x
    – clt60
    Jun 6, 2011 at 16:18
  • Ah, I see, you're mounting the directory and rsync ing locally! I was trying to rsync from the Mac to the server! I will try this and get back to you.
    – Josh
    Jun 6, 2011 at 16:50
0

I don't know why, but the connection to the server was being broken during the file copy and I am assuming it was because the high amount of data being transferred at such high speed was more than the backup server could handle; it would become unresponsive for a short period and the Mac would forcibly disconnect the DMG, and the backup would fail.

I found a solution: before backing up, in System Preferences, I lowered my Ethernet NIC's speed from 1000Mbps to 100Mbps. Then, the backup seemed to work flawlessly, because the data rate was constrained and the backup server didn't get overwhelmed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .