I need to be able to copy about 2TB of data from an external drive (single USB external disk) to a another external drive (Drobo attached via Firewire).

Finder is not an option. If it hits any problem, it stops the process and I have to figure it out why it failed and start over. Could take me months to get through it.

I've read about CP and RSYNC but I'm not too familiar and I'm afraid I might get some of the switches wrong and miss files, or screw up the file dates, or worst...

Can anyone suggest how I might pull this off with minimum risk and cost?

Thank you.

  • Starting with MacOS 10.13.4+, you actually can use Finder to copy huge files (and all the associated links). I just tried it myself to create another Time Machine backup disk (from an existing one) and it works quickly and beautifully. Apparently, before 10.13.4, this was a bug but that's been fixed. More info in this answer: apple.stackexchange.com/a/323691/261070. – youngrrrr Aug 31 at 2:48
up vote 127 down vote accepted

remote sync, rsync, is a reliable choice for copying large amounts of data. You can prepare the command and perform a dry-run before committing to the copy; add --dry-run to simulate the copy.

Your final command will be fairly simple:

sudo rsync -vaE --progress /Volumes/SourceName /Volumes/DestinationName

The flags are:

  • v increases verbosity.
  • a applies archive settings to mirror the source files exactly, including symbolic links and permissions.
  • E copies extended attributes and resource forks (OS X only).
  • progress provides a count down and transfer statistics during the copy.

sudo, is used to ensure rsync has appropriate rights to access and read all files on your drive regardless of owner. This also allows rsync to write the files to the new drive recreating the original owner information.

rsync is likely the best choice because it can be rerun in case of problems, offers detailed logging, and is as fast as can be while remaining safe.

There are numerous guides for getting the most from rsync, rsync command examples provides relevant examples. As @GordonDavisson comments below, take care with the trailing slashes; these can make a world of difference if your copy starts with a folder.

Alternative tools include ditto and cp. Both very reasonable choices but offering differing syntax.

  • Does the example command properly handle transfers of Time Machine backups? – Swisher Sweet Nov 8 '16 at 16:25
  • 3
    @SwisherSweet you should consider asking a new question. In the mean time, follow Apple's advice for transferring a Time Machine back-up. – Graham Miln Nov 8 '16 at 16:30
  • OSX comes with an outdated version of rsync which does not support all the switches that the GNU version has. There is a more modern version in homebrew which can be installed with brew install https://raw.github.com/Homebrew/homebrew-dupes/master/rsync.rb – ccpizza Nov 13 '16 at 22:33
  • Error: bad URI(is not URI?): https://raw.github.com/Homebrew/homebrew-dupes/master/rsync.rb Please report this bug: https://git.io/brew-troubleshooting :( – Henno Jan 2 '17 at 20:56
  • 1
    Does the "more modern version" include the switches that Apple added to support HFS+ and resource forks? – WGroleau Mar 23 at 1:34

I used bittorrent sync to sync 8TB between my Drobo and QNAP.

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

  • bittorrent sync is a way to do this as well. The downvotes are (in my opinion) not realy needed. – CousinCocaine Oct 25 '15 at 13:19
  • 3
    @CousinCocaine yeah, but the question was about how to do something, not what app to use. – Patience Jan 1 '17 at 4:18

I answered a similar question here a while back. My answer is copied below.


I use cp -Rfv sourcefile destinationfile with success on a pretty regular basis.

cp = copy

R = maintains file hierarchies

f = if an existing destination file cannot be opened, remove it and try again

v = verbose mode, displays files transferred as it progresses

sourcefile = data you want to copy

destinationfile = directory/drive you want to copy to

  • 8
    It's worth noting however that cp will recopy files even if they haven't changed whereas rsync will not waste time copying files that already exist on the target. Thus you can restart from where you left off and/or do incremental copying. – David Jul 3 '15 at 22:53

The "fastest" way would be to physically move both drives to be internal to a single computer, do the copy (or rsync), and then move them back.

I'd still use rsync, because if interrupted for any reason (cat steps on the power switch?), rsync will not recopy the files that were already successful. It also won't copy any files that are the same and in the same place.

  • Note that this is faster because of the size of the data set. If it were a lot smaller, the speed advantage would be canceled by the time moving the drives. – WGroleau Mar 23 at 15:35
  • No, faster for a large amount of data. For a small amount of data, the copy is still faster, but the time it takes to move the drives cancels that out. However, it's potentially faster under NON-perfect conditions, because if interrupted, rsync is able to avoid copying again the ones that were done before the interruption. – WGroleau Sep 7 at 18:12
  • If the transfer speed is faster but an interruption makes you start over, the entire process is likely to not be faster, and the entire process is what matters. "perfect conditions" = nothing interrupts the process. In my experience it's very common for something to happen to cause a problem when you're dealing with a massive transfer. That's why I refer to an uninterrupted transfer as "perfect". – iconoclast Sep 13 at 21:12

If you don't want to go the command line route, I use the FreeFileSync app routinely to sync 2 TB of data from an external array to a network location without issue. You can control how it handles errors and get a log when its complete.

If you would like to copy drive A to drive B exactly you can do this with Disk Utility.


We will be using Disk Utility's restore function. Some background on the different between Restoring vs copy and pasting:

The Restore function in Disk Utility makes use of a block copy function that can speed up the copy process. It also makes an almost exact copy of the source device. When we say "almost exact," we don't mean to imply that useful data may get left behind, because that's not the case. What it means is that a block copy copies everything in a data block from one device to the other. The results are almost an exact copy of the original. A file copy, on the other hand, copies data file by file, and while the file data remains the same, the location of the file on the source and destination devices will likely be very different.

Using a block copy is faster, but it does have some limits that affect when it can be used, the most important being that copying block by block requires that both the source and destination devices be first unmounted from your Mac. This ensures that block data doesn't change during the copy process. Don’t worry, though; you don’t have to do the unmounting. Disk Utility’s Restore function takes care of that for you. But it does mean that neither the source nor the destination can be in use when you use the Restore capabilities.

Source: https://www.lifewire.com/use-disk-utility-to-clone-macs-drive-4042367


Instructions

for copying all files from Device A to Device B

WARNING: When you restore one volume to another, all the files on the destination volume are erased. Before you restore a volume, copy any files on the destination volume that you want to save to a different volume.

  1. Choose View > Show All Devices.
  2. Select the volume in the sidebar you want to restore, then click the Restore button enter image description here or choose Edit > Restore.

This is the volume that is erased and becomes the exact copy. [Device B]

  1. Click the Restore pop-up menu, then choose the volume you want to copy [Device A].
  2. Click Restore, then click Done.

Source: https://support.apple.com/guide/disk-utility/restore-a-disk-dskutl14062/mac

  • Block copy also wstees time copying free space and will fail if the destination is one block smaller than the source. (Might fail in a way that isn't detected immediately.) And if the destination is bigger, the extra space becomes unusable. – WGroleau Sep 13 at 22:09
  • @WGroleau can you expand on "the extra space becomes unusable" – JBis Sep 13 at 23:53
  • block copy includes the block that says which block is the last. If the destination is big enough for the data but smaller than the source, the operating system thinks there are blocks available that don't exist. If the destination is bigger, the O.S. thinks the extra blocks don't exist. – WGroleau Sep 13 at 23:57
  • @WGroleau just so I understand you correctly if I have a 500gb drive with a single 1gb file and block copy to a 1000gb drive what will happen? And can you site link? – JBis Sep 14 at 0:03
  • Look at the man page for ‘dd’. It will copy 500GB, but most of the 499 GB will be empty. The part that’s not is only visible to the operating system, and tells it that you have a 500 GB drive. Also look up “partition map” and “volume map” – WGroleau Sep 14 at 16:32

protected by Community Oct 28 '16 at 1:36

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.