13

I'm trying to move my time machine backup files all under Backups.backupdb to another drive. I initiated a file copy overnight (b/c I saw that it took OSX forever to prepare for the copy...it basically was counting the files for hours). In the morning I saw that only certain backups(folders with dates) got copied over. I then tried to copy over the ones that didn't get copied...but the OS wouldn't allow me to do that. I got and error that "The operation can’t be completed because backup items can’t be modified." So my plan is to delete the incomplete copy on the new drive and then try to copy over the Backups.backupdb folder again.

Pretty frustrating. Is there a faster way to copy these files via a terminal command so that it doesn't perform all of that file counting prep?

I probably can tar up the entire folder and then do a copy, but will that interfere with any of the file permissions, etc.? The one thing with this approach is that I don't have any more space on my source volume for the tar.

UPDATE

I've tried some of the methods that people have suggested below, specifically using Disk Utility's restore function and it's giving me some error messages and unexpected results (at least to me). I've tried to do the restore two ways:

  • With "Erase Destination" checked : Each time (I've tried twice), when the restore has finished I see a message "Could not restore - Invalid Operation" and "Could not restore - Invalid Argument". However, my destination disk does get a copy of my TM files. The weird thing is that my destination disk is EXACTLY like my source disk...even the size. My destination disk is actually 1 TB but after the restore, it shows as 200 GB when I get info from the finder. But in Disk Utility, it shows a 1 TB partition!

I then tried to verify/repair the disk and got:

    Invalid B-tree node size
    Checking Journaled HFS Plus volume.
    Invalid B-tree node size
    Volume repair complete.
    Updating boot support partitions for the volume as required.
    Error: Disk Utility can’t repair this disk. Back up as many of your files as possible, reformat the disk, and restore your backed-up files.

Don't know if I'm even suppose to verify/repair a TM disk...

  • With "Erase Destination" UNchecked : The restore never starts and I get:
    Could not restore - Operation not permitted
11

A normal copy (or copy via rsync or ditto) will not replicate a Time Machine fully as it will convert two directories linked together (as occurs in successive TM backups with no change between) into two separate directories.

The best way is to copy the whole the disk using Disk Utility or the block copy part of Carbon Copy Cloner and probably similar on SuperDuper.

  • 1
    From the ditto man page: "ditto preserves file hard links (but not directory hard links) present in the source directories" so no help here. It's either Disk Utility or a tool like SuperDuper or CCC. – nohillside Jan 1 '12 at 12:20
  • @patrix Thanks - the web man page says nothing on that - CCC uses ditto or rsync for copying so will only do this if it does a block copy help.bombich.com/kb/troubleshooting/… – Mark Jan 1 '12 at 21:33
  • My Source disk only contains the Time Machine backup. My Destination disk contains other files. I don't want a clone of my Source disk. I just want to copy the Time Machine files to the Destination disk. – milesmeow Jan 2 '12 at 11:44
  • 3
    After many attempts to copy my TM files to a new disk, Disk Utility and Carbon Copy Cloner both DID NOT do the trick. SuperDuper did it perfectly on the first run and didn't reduce the size of my destination partition! – milesmeow Jan 5 '12 at 15:51
  • 2
    Another vote for SuperDuper! here. v3.2.4 successfully copied a big Time Machine backup folder to a new disk, under macOS 10.14.2 Mojave, without it taking any more space. (Which Finder could not do…) Time Machine happily continues using the new disk as if it were the old one. – gidds Jan 28 at 22:14
4

Migrating a full 3TB Time Machine encrypted drive to a new 8TB one on macOS 10.14, I ran into all sorts of issues. Trying to do a restore in Disk Utility errored out with “unable to validate source” or “Operation not permitted.” Trying some other suggestions in this post and others, I was able to get exciting new error messages like “Catalog file on image/volume is too badly fragmented,” but no copy.

What worked in the end, at the terminal:

  1. Erase the new disk with Disk Utility, matching the format of the source drive: MacOS Extended (Journaled, Encrypted)
  2. Use diskutil cs list in the terminal to get the exact byte size of the Logical Volume on the old drive, and the GUID of the new Logical Volume, as well as the disk numbers for both, e.g., disk4.
  3. Use the exact byte size from step2 as the size of the new volume. In my case with a 3TB drive it was 2,999,772,905,472 bytes:

    sudo diskutil cs resizeVolume $new_lv_guid 2999772905472
    
  4. Using the pv command from homebrew, do a low-level block copy of the disks. This is a lot like using dd, except you get a progress meter with ETA.

    You need to get the disk numbers from the diskutil cs list output. Be careful. It’s very easy to accidentally overwrite your full backup drive with the new blank one here.

    sudo sh -c "$(which pv) --buffer-size 50M -s 2999772905472 < /dev/rdisk${source} > /dev/rdisk${target}"
    

    If you get a permission denied / operation not permitted error here, go into Security & Privacy Preferences and add Full-Disk Access for Terminal.app.

    For me this took about 10 hours—I let it run overnight—but, with pv, at least you get a progress meter with an ETA.

  5. Now, expand the volume to take up all remaining space on the drive:

    sudo diskutil cs resizeVolume $new_lv_guid 0
    

    This took ~ 3 hours for me, with about 5 years of backups. Most of that time was spent by macOS fscking.

Now you can enjoy your new, more spacious Time Machine drive. You can repurpose the old one, or stow it somewhere safe in case something happens to the new drive.


The resizing steps seem to be important; skipping them resulted in a 10-hour file copy that yielded an 8TB volume containing a 3TB filesystem that I couldn’t figure out how to resize.


UPDATE One potential downside to this approach is that because it’s a bit-for-bit copy, the identifiers are the same between the old disk and the new disk. If I connect the old full disk, Time Machine thinks it’s the new disk, tries to back up, and starts deleting old backups to make room for new ones. It seems like a fine approach for moving data to a larger disk where the older smaller disk will then be wiped.

  • Hello Andrew! Thanks for taking the time to type out this step-by-step guide (and I'm hoping to use it to transfer my 1TB backup to a 4TB disk, which up until now hasn't been successful because Finder-copied folders & files take up so much more space on the new disk than the original one). My question to you is: can I do these steps without cs a.k.a. corestorage enabled? Enabling core storage appears to be a potentially unnecessary PITA, but it may be necessary because of the guid step 3. – Michael Dautermann Dec 19 '18 at 3:41
  • @MichaelDautermann Core Storage is necessary for FileVault which is extremely highly recommended for backup drives, in order to protect your privacy in cases of loss, theft, or improper disposal. – andrew Dec 19 '18 at 14:25
2

Why not just use terminal:

cp -RnpP Backups.backupdb
  • -R recursive
  • -n do not overwrite (if existing copy remnants remain from previous attempt)
  • -p preserve ACL's, permissions, creation/mod dates, etc.
  • -P preserve hard links, do not follow any hard or symlinks.
  • This is not true. Read man cp for macOS. The regular cp command shipped with macOS does not copy hard links with -P. The man page actually says "Note that cp copies hard-linked files as separate files. If you need to preserve hard links, consider using tar(1), cpio(1), or pax(1) instead." – chmac Jan 22 at 11:57
0

This answer will not get it done faster, but I've found is a way to copy the data properly while preserving the de-duplication (hard links) and permissions. As an added bonus I use this to make a compressed dmg of the final product for archiving.

  1. Using Disk Utilities, make a disk image that is larger then your Backups.backupdb directory. I would also suggest that you use sparse bundle disk image for Image Format and Hard disk for Partitions. After this image is mounted, Get info on it and unselect Ignore ownership on this volume.

  2. Now turn off Time Machine, and using the finder copy the Backups.backupdb folder to the mounted image. The finder will ask you for super user permissions to copy the data. Get a drink or do something else for a while.

  3. When the copy is finished, look to be sure everything is okay and unmount the image. From Disk Utility, select Convert and turn the sparse bundle image into a compressed image. Again, this may take a while.

You should end up with two copies of your Time Machine backup, you can delete the sparse bundle version, and put the dmg in a safe place as an archive in time.

One thing I haven't tried with this is to do a system restore from the dmg, but I suspect that it should work, my goal was more for archiving the time machine incremental changes and keeping the hard link structure.

I have also tried rsync and cp, but they didn't seem to keep the hard link structure which would end up making x times the size, x being the amount of dates you had in the past. This method worked well, but again may not get speed of a block copy solution.

0

Apple has an official tutorial for this: “Time Machine: How to transfer backups from a current backup drive to a new backup drive”.

The high-level steps from that page:

  1. Check the format of your new backup drive
  2. Set permissions on your new backup drive
  3. Temporarily turn Time Machine off
  4. Copy your backup data from your original drive to your new drive
  5. Set Time Machine to use your new drive

This is how the page recommends doing the copying step:

Copy your backup data from your original drive to your new drive

  1. Open a new Finder window. In the Finder sidebar, click the icon of the original backup drive.
  2. Open a new Finder window. In the Finder sidebar, click the icon of the new backup drive.
  3. Drag the folder "Backups.backupdb" from the original backup drive to the top level of the new backup drive.
  4. Enter an administrator name and password, then click OK to start the copying process.

Copying your backup data might take some time to complete, depending on the size of your backup.

  • 5
    I for one am looking at this question because following that tutorial (which suggests copying the backup folder with Finder) and leaving it running overnight, it ended on some permission issue with about 500/940gb copied. I then did a sudo rsync last night, but this morning find ERROR: out of memory in flist_expand [sender] and my copy is now ~600gb. I haven't decided what to do next, but suspect most people reading are already aware of the official tutorial. – PeterT Dec 13 '14 at 9:00
  • @PeterT I have just tried the tuto too and got the same issue as you. I am not sure anyone knew about the tutorial, otherwise someone would have mentioned it here and the outcome following it. Now, people know it is not worth trying. – David Andreoletti Dec 14 '14 at 9:39
  • 1
    Using finder to copy the folder takes an age to build the file list and then fails anyway with not enough disk space so must be miscalculating. – malhal May 5 '17 at 9:13
  • 1
    That's exactly my problem. The original TM volume is 550GB, the new one was 600GB. Still Mojave complained about insufficient space on the volume. I'm using now SuperDuper! in "Backup - all files" mode. – Markus Rudel Oct 3 '18 at 6:24
  • 1
    Apple's tutorial failed for me, in macOS Mojave 10.14.2. I tried to copy a 3TB backup archive to an 8TB drive; Finder spent nearly 5 days copying (saying ‘5 seconds left’ for most of that), before giving up and complaining that the drive was full! And it was — even though it had only copied about 2/3 of the backups. Clearly, it's not preserving hardlinks but creating fresh copies of each. So this answer is not currently correct. – gidds Jan 25 at 20:28
-1

rsync is a great utility for stuff like this. I generally use it for stuff like this. In this case I might use the -aP flags. I think part of -a ("archive") is also to preserve permissions, ACLs and the like, but I'm not sure.

IIRC, there's also a --delete option that allows you to delete the source file once it has been successfully copied to the destination. I'd be wary of using that though - usually I do a complete mirror without the --delete option, then I'll re-run the command with the -c and --delete options. -c is checksum, so it checks all the files you've downloaded against all those on the source via checksum, then deletes the source if there's a match, otherwise it re-copies or resumes copying as the case may be.

  • 5
    rsync doesn't maintain hard links on directories. Copying a while TM backup will duplicate a lot of directories – nohillside Jan 1 '12 at 19:41
  • 1
    @patrix - I can confirm this. I've tried it. Directory hard-links are nearly unique to HFS+, and rsync does not understand them. – Fake Name Jan 2 '12 at 12:22
  • 2
    -H, --hard-links preserve hard links – Pete Ashdown Jun 4 '17 at 21:23
-2

With hard drives, when you move multiple files from one drive, the reader moves back and forth making a scary clicking noise, and it slows the transfer rate significantly, for example- one file with usb 2.0 moves at 30 mbps on my computer from 2 external harddrives, but 2 files move at 11 mbps. and 3 files move at 6 mbps. etc etc. zip files will move faster than files.

  • 2
    How does this answer the OP's question? – fsb Aug 12 '16 at 14:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .