How can I back-up an external drive while preserving attributes, folder creation date and showing progress?

Question

Final goal

• Create a copy of all files one external hard drive to another external hard drive, both formatted as HFS+.
• Preserve the folder creation date.
• Preserve finder colour label.
• During the copy I want to see some kind of progress indicator.

Using cp

If I copy using

cp -a /origin/folder/ /destination/folder

this only preserves file creation dates, not folder creation dates.

Using rsync 3.1.3

If I copy using rsync 3.1.3 (installed via Homebrew)

rsync --recursive --info=progress2 -hhh --xattrs --times --crtimes /origin/folder /destination/folder 

this preserves folder creation dates, colour labels, but riddles the output with below errors (I can't hide them with the --quiet option), so I cannot keep track of the progress

rsync: get_xattr_names: llistxattr("/origin/folder/filename",1024) failed: Operation not permitted (1)

Answer 1

Just after I posted this question I found a solution.

I noticed that the rsync: get_xattr_names errors all had one thing in common: the files that raised the error where always ._ files.

I read that ._ files are used to store information that would go into a HFS+ extended attribute. As I'm copying between HFS+ drives I figured I don't need these files.

So I added an --exclude argument to my rsync command which excludes all filenames that start with ._

rsync --exclude="._*" --recursive --info=progress2 -hhh --xattrs --times --crtimes /Volumes/origin/ /Volumes/destination

The command:

• Filters ._ files beforehand (--exclude="._*"), preventing get_xattr_names errors.
• Preserves the folder creation dates via the --times --crtimes argument.
• Preserves Finder colour labels via the --xattrs argument.
• Shows progress in a human readable format via the --info=progress2 -hhh argument.
• (Added bonus: preserves custom folder icons as well, via the --xattrs argument)

Answer 2

In your answer, you write, "I read that ._ files are used to store information that would go into a HFS+ extended attribute. As I'm copying between HFS+ drives I figured I don't need these files."

I would not make this assumption without further testing. It is not necessary that the resource fork information captured in these existing dot bar files has been (re-)incorporated (back) into the associated file.

There is a Wikipedia article on "AppleSingle and AppleDouble formats":

https://en.wikipedia.org/wiki/AppleSingle_and_AppleDouble_formats

The "dot bar" files can be created under a number of cases, for example, see:

Why are dot underscore ._ files created, and how can I avoid them?

The question whether the dot bar files should be re-incorporated was discussed here:

How should I reconcile dot-underscore files after a manual backup?

The OS X/macOS command dot_clean has a number of options regarding how to deal with them:

--keep=mostrecent

The default option. If an attribute is associated with a data fork, use that. Otherwise, use information stored in the AppleDouble file. Note that the native fork's data is preferred even if the data in the AppleDouble file is newer.

--keep=dotbar

Always use information stored in the AppleDouble file, replacing any extended attributes associated with the native file.

--keep=native

Always use the information associated with the data fork, ignoring any AppleDouble files.

You can consult man dot_clean for further information and options.

You might remove some uncertainty by determining whether the files with an associated dot bar have any resource fork information attached without reference to the dot bar files. This might be somewhat involved to test, as copying the files might re-incorporate the dot bar files during the copy.

You may not care, or may not care to spend the time to determine, and simply run dot_clean with its default option.

Or, you could ignore the issue for now and not exclude the dot bar files during your rsync backups.

It should be pointed out that much of this discussion is in regards to how to treat the source/origin of the backup. Presumably, the purpose of a backup is to duplicate as exactly as possible the source. As such, I would not exclude the dot bar files as part of your backup. Instead, I would look to the reason the dot bar files exist in your source. Then, whether, and later how, to "fix" them.