4

Final goal

  • Create a copy of all files one external hard drive to another external hard drive, both formatted as HFS+.
  • Preserve the folder creation date.
  • Preserve finder colour label.
  • During the copy I want to see some kind of progress indicator.

Using cp

If I copy using

cp -a /origin/folder/ /destination/folder

this only preserves file creation dates, not folder creation dates.

Using rsync 3.1.3

If I copy using rsync 3.1.3 (installed via Homebrew)

rsync --recursive --info=progress2 -hhh --xattrs --times --crtimes /origin/folder /destination/folder 

this preserves folder creation dates, colour labels, but riddles the output with below errors (I can't hide them with the --quiet option), so I cannot keep track of the progress

rsync: get_xattr_names: llistxattr("/origin/folder/filename",1024) failed: Operation not permitted (1)
3

Just after I posted this question I found a solution.

I noticed that the rsync: get_xattr_names errors all had one thing in common: the files that raised the error where always ._ files.

I read that ._ files are used to store information that would go into a HFS+ extended attribute. As I'm copying between HFS+ drives I figured I don't need these files.

So I added an --exclude argument to my rsync command which excludes all filenames that start with ._

rsync --exclude="._*" --recursive --info=progress2 -hhh --xattrs --times --crtimes /Volumes/origin/ /Volumes/destination

The command:

  • Filters ._ files beforehand (--exclude="._*"), preventing get_xattr_names errors.
  • Preserves the folder creation dates via the --times --crtimes argument.
  • Preserves Finder colour labels via the --xattrs argument.
  • Shows progress in a human readable format via the --info=progress2 -hhh argument.
  • (Added bonus: preserves custom folder icons as well, via the --xattrs argument)
| improve this answer | |
  • Note also -a, --archive - archive mode; same as -rlptgoD (no -H). When I use it (macOS and Linux) I always specify -a but even though it includes -t I could swear that I have seen it not preserve times so I always specify it explicitly. – Pryftan Mar 1 at 21:42
  • Thanks, I was aware of the archive option, but felt it I would better understand the command when using (and reading about) each argument separately. When you say seen it not preserve times, do you mean the creation time? If so, that's as expected, as only the (mac only?) --crtimes argument takes care of that. – Saaru Lindestøkke Mar 1 at 22:23
  • No. I meant modification/change times. Under Linux few filesystems even have a creation time. And I totally understand wanting to get all the options down. It's a good habit to get into esp if you're new to the command! Actually that's how I started it but that was a very long time ago now. (I do this for macOS too though.) Oh and have a +1 for the question and answer. Further I know at least on some SE sites it's perfectly acceptable to accept your own answer. Maybe that's not the case for Apple SE but just thought I'd mention it. – Pryftan Mar 5 at 12:51
  • One more thing is that on Homebrew be aware that it takes ownership (your user) of the system directory. That's a bad thing. In case you weren't aware of that. That's why I use MacPorts. It doesn't have everything that Homebew has but for me at least I am an old-timer and remember before the package managers (and older Unices than Linux for that matter) and also a programmer so I can compile things I need in those cases. Cheers. – Pryftan Mar 5 at 12:54
  • ownership of the system directory? Can you elaborate on that? I found this question that indicates something else, but maybe that has changed again. I started with macports, but the number of packages was limiting, that's why I switched to homebrew. – Saaru Lindestøkke Mar 5 at 13:12
0

In your answer, you write, "I read that ._ files are used to store information that would go into a HFS+ extended attribute. As I'm copying between HFS+ drives I figured I don't need these files."

I would not make this assumption without further testing. It is not necessary that the resource fork information captured in these existing dot bar files has been (re-)incorporated (back) into the associated file.

There is a Wikipedia article on "AppleSingle and AppleDouble formats":

https://en.wikipedia.org/wiki/AppleSingle_and_AppleDouble_formats

The "dot bar" files can be created under a number of cases, for example, see:

Why are dot underscore ._ files created, and how can I avoid them?

The question whether the dot bar files should be re-incorporated was discussed here:

How should I reconcile dot-underscore files after a manual backup?

The OS X/macOS command dot_clean has a number of options regarding how to deal with them:

--keep=mostrecent

The default option. If an attribute is associated with a data fork, use that. Otherwise, use information stored in the AppleDouble file. Note that the native fork's data is preferred even if the data in the AppleDouble file is newer.

--keep=dotbar

Always use information stored in the AppleDouble file, replacing any extended attributes associated with the native file.

--keep=native

Always use the information associated with the data fork, ignoring any AppleDouble files.

You can consult man dot_clean for further information and options.

You might remove some uncertainty by determining whether the files with an associated dot bar have any resource fork information attached without reference to the dot bar files. This might be somewhat involved to test, as copying the files might re-incorporate the dot bar files during the copy.

You may not care, or may not care to spend the time to determine, and simply run dot_clean with its default option.

Or, you could ignore the issue for now and not exclude the dot bar files during your rsync backups.

It should be pointed out that much of this discussion is in regards to how to treat the source/origin of the backup. Presumably, the purpose of a backup is to duplicate as exactly as possible the source. As such, I would not exclude the dot bar files as part of your backup. Instead, I would look to the reason the dot bar files exist in your source. Then, whether, and later how, to "fix" them.

| improve this answer | |
  • Thanks for the extensive write-up. In your last paragraph you say I would look to the cause of these issues in your source. Which issues is that? That there's no attribute that rsync can get from the ._ files? – Saaru Lindestøkke Mar 6 at 19:29
  • The existence of the ._ files themselves. The HFS+ file system can "integrate" resource forks natively. The dot bar files are typically used as a mechanism to transfer the resource fork to a file system without native support. Why has the resource fork not been integrated during the creation of the associated files? Indeed, do the associated files have a resource fork? And, if so, does the resource fork match the information in the associated dot bar files? (Made edits to answer to help clarify.) – newyork10023 Mar 6 at 19:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .