Skip to main content
revision aimed at improving clarity
Source Link
Seamus
  • 5.1k
  • 3
  • 23
  • 44

"I have discovered the hard way that Apple's "Migration Assistant" doesn't always preserve file timestamps"

"I have discovered the hard way that Apple's "Migration Assistant" doesn't always preserve file timestamps"

commentComment: It seems that has become Apple's MO - you must discover all things the hard way"hard way".

Caveats: I'm not entirely sure this will resolve your issue or answer your question, but I hope it will help. I don't actually know what Migration AssistantMigration Assistant does because in 12 years and 5 different Macs, I haveas I've never used it. So this answer may be a partial answer, or it may be incompatible with Migration Assistant. As a guessHowever, I'd say use rsync after Migration Assistant - to clean upbased on the mess it has left. In other words, rsync can likelyexample below, I've shown restore not only theit will preserve your timestamps lost by MA, it can also restore all file(and other metadata thatyou may have been lost. Others here may have better ideas; my answer will be limited to rsync).

I used rsync versionver 3.2.4 for this answer, installed from MacPorts. I do not know if thecan't guess what features your version of rsync version included with your version of macOS includes the required options shown here - you may wish to investigate that instead of installing a current version of. You should consider upgrading rsync with MacPorts or Homebrew. Apple hasn't upgraded "their" version of rsync in many years, and it lacks the ability to support the methods and options used here.

Background: I've begunI began using rsync as my primary backup tool for macOS recently, andsome time ago because I had to address some of the same issues as you: mangled and missing metadata. In my case, the issue was causedperhaps exacerbated by crossing a file system boundarycrossing a file system boundary - from APFS on my local drive to btrfs via SMB on a Synology NAS'Synology NAS'. In your case - Migration Assistant, using -Migration Assistant, I cannot even imagine how itApple could not even get the timestamp metadata correct, - but you're certainly not the only one who's reported such issuesthis issue.

I made some notes while working to resolve my issue, and they morphed into a "recipe" on my macOSin this GitHub repo. I'll try to keep this answer rather brief sinceas most of the details are covered in that document.

rsync with metadata preservation


Q: Can I use rsync to fix the timestamps that Migration Assistant trashed?

A: Yes - as shown below and verified

rsync with metadata preservation

rsync effectively copies data from a source (the "from" location) to a destination (the "to" location). If you are using a fileserver (NAS) as an intermediary betweena backup for your two MacsMac, these commands should do whatthe GitHub recipe will provide complete guidance to make and restore an rsync backup without losing metadata (attrs & xattrs).

However, as you need. I don't see any only to "migrate" files & folders from an old Mac to a new Mac, there is no need to involve an intermediate NAS. Consequently, but it maythe GitHub recipe can be simplified.

The scenario below assumes that the remote folders have been mounted (perhaps via convenient'File Sharing' or desirablein 'System Settings'). Alternatively, if you want a backup. You should be ableprefer to accomplish theenable 'Remote Login' you can run rsync operations over an SSH connection; thevia SSH.

The following commands illustratecommand code block shows the command syntax & options needed regardless of whether you use the intermediary NAS or notfor rsync to perform an old Mac-to-new Mac "migration":

from "old Mac" to NAS:

SCENARIO: from "oldMac" to "newMac"

Assuming the source & destination folders are:
SOURCE:SOURCE: /Users/MyHome/MyDataoldMacFolder/
DESTINATION:DESTINATION: /System/Volumes/Data/mntUsers/MyNASMyHome/newMacFolder
From a terminal on your "old Mac":

% SRC-FLDR="/Users/MyHome/MyData/"
% DST-FLDR="/System/Volumes/Data/mnt/MyNAS/"
% rsync -rlAXtgoDivv --dry-run --fake-super $SRC-FLDR $DST-FLDR > rsync.log 2>&1

Explanation: This rsync command uses the --dry-run option, and therefore will not actually move or modify any files. It will give you a detailed log (the ivv options) of what files would have been moved or modifiedFrom Terminal.app on your "old Mac":. Once you are happy with the results, simply remove the --dry-run option, and run again for effect. Review the rsync.log file using the "Decoding Table"

from NAS to "new Mac":

SOURCE: /System/Volumes/Data/mnt/MyNAS/
DESTINATION: /Users/MyNewMacHome/MyData/
From a terminal on your "new Mac":

% SRC-FLDR="/System/Volumes/DataUsers/mntMyHome/MyNASoldMacFolder/"
% DST-FLDR="/Users/MyNewMacHome/MyDataMyHome/"newMacFolder"
% rsync -rlAXtgoDivv --dry-run -M--fake-super $SRC-FLDR $DST-FLDR > rsync.log 2>&1

from "old Mac" to "new Mac"

If you can make an SSH connection between your "old Mac" & "new Mac", it's not necessary to involve an intermediate NAS. The same rsync options may be used; I'll leave it to you to formulate the SOURCE and DESTINATION folders.Notes:

  1. --dry-run this option prevents rsync from actually copying any files! It's used so that you can see what will be copied once the option is removed.

  2. If you've not set up file-sharing to mount the remote folder(s), you may use an SSH connection by adding the option -e ssh to the rsync command. This will require that you've set up 'Remote Login' in 'System Settings'.

Verifying Results - Preservation of All Metadata:

The GitHub recipeGitHub recipe includes a short zsh script that will stat all of the files, folders, links, etc between the SOURCE and DESTINATION folders. It may be run followingafter the rsync operations to verify that at least the chosen stat attributes are the samehave been preserved.

The rsync options presented here will do more than preserve the metadata containing the date-time stamps; they will also preserve all extended attributes (xattrs) . The GitHub recipe also includes a simple script to compare the output of xattr -lrsvx for all files (that have xattrs) in the SOURCE and DESTINATION directories.

The stat utility that I use here is the GNU coreutils version 9.1. The find version I use is the GNU findutils version 4.9.0. Both are readily available through MacPorts, and available (and likely through Homebrew) for virtually every version of macOS. The xattr command is a macOS utility.


References and Further Reading:

  1. There’s more to files than data: Extended Attributes

  2. From quarantine to provenance: extended attributes

  3. From quarantine to provenance: how xattrs are copied

  4. Extended attributes

  5. How to use rsync over ssh

"I have discovered the hard way that Apple's "Migration Assistant" doesn't always preserve file timestamps"

comment: It seems that has become Apple's MO - you must discover all things the hard way.

Caveats: I'm not entirely sure this will resolve your issue or answer your question, but I hope it will help. I don't actually know what Migration Assistant does because in 12 years and 5 different Macs, I have never used it. So this answer may be a partial answer, or it may be incompatible with Migration Assistant. As a guess, I'd say use rsync after Migration Assistant - to clean up the mess it has left. In other words, rsync can likely restore not only the timestamps lost by MA, it can also restore all file metadata that may have been lost. Others here may have better ideas; my answer will be limited to rsync.

I used rsync version 3.2.4 for this answer, installed from MacPorts. I do not know if the rsync version included with your version of macOS includes the required options shown here - you may wish to investigate that instead of installing a current version of rsync.

Background: I've begun using rsync as my primary backup tool for macOS recently, and had to address some of the same issues as you: mangled and missing metadata. In my case the issue was caused by crossing a file system boundary - from APFS on my local drive to btrfs via SMB on a Synology NAS. In your case - Migration Assistant - I cannot even imagine how it could not even get the timestamp metadata correct, but you're certainly not the only one who's reported such issues.

I made some notes while working to resolve my issue, and they morphed into a "recipe" on my macOS GitHub repo. I'll try to keep this answer rather brief since most of the details are covered in that document.

rsync with metadata preservation

rsync effectively copies data from a source (the "from" location) to a destination (the "to" location). If you are using a fileserver (NAS) as an intermediary between your two Macs, these commands should do what you need. I don't see any need to involve an intermediate NAS, but it may be convenient or desirable if you want a backup. You should be able to accomplish the rsync operations over an SSH connection; the following commands illustrate the syntax & options needed regardless of whether you use the intermediary NAS or not:

from "old Mac" to NAS:

Assuming the source & destination folders are:
SOURCE: /Users/MyHome/MyData/
DESTINATION: /System/Volumes/Data/mnt/MyNAS/
From a terminal on your "old Mac":

% SRC-FLDR="/Users/MyHome/MyData/"
% DST-FLDR="/System/Volumes/Data/mnt/MyNAS/"
% rsync -rlAXtgoDivv --dry-run --fake-super $SRC-FLDR $DST-FLDR > rsync.log 2>&1

Explanation: This rsync command uses the --dry-run option, and therefore will not actually move or modify any files. It will give you a detailed log (the ivv options) of what files would have been moved or modified. Once you are happy with the results, simply remove the --dry-run option, and run again for effect. Review the rsync.log file using the "Decoding Table"

from NAS to "new Mac":

SOURCE: /System/Volumes/Data/mnt/MyNAS/
DESTINATION: /Users/MyNewMacHome/MyData/
From a terminal on your "new Mac":

% SRC-FLDR="/System/Volumes/Data/mnt/MyNAS/"
% DST-FLDR="/Users/MyNewMacHome/MyData/"
% rsync -rlAXtgoDivv --dry-run --fake-super $SRC-FLDR $DST-FLDR > rsync.log 2>&1

from "old Mac" to "new Mac"

If you can make an SSH connection between your "old Mac" & "new Mac", it's not necessary to involve an intermediate NAS. The same rsync options may be used; I'll leave it to you to formulate the SOURCE and DESTINATION folders.

Verifying Results:

The GitHub recipe includes a short zsh script that will stat all of the files, folders, links, etc between the SOURCE and DESTINATION folders. It may be run following rsync operations to verify that at least the chosen stat attributes are the same.

The rsync options presented here will do more than preserve the metadata containing the date-time stamps; they will also preserve all extended attributes. The GitHub recipe also includes simple script to compare the output of xattr -lrsvx for all files (that have xattrs) in the SOURCE and DESTINATION directories.

The stat utility that I use here is the GNU coreutils version 9.1. The find version I use is the GNU findutils version 4.9.0. Both are readily available through MacPorts, and available for virtually every version of macOS. The xattr command is a macOS utility.

"I have discovered the hard way that Apple's "Migration Assistant" doesn't always preserve file timestamps"

Comment: It seems that has become Apple's MO - you must discover all things the "hard way".

Caveats: I don't actually know what Migration Assistant does as I've never used it. However, based on the rsync example below, I've shown it will preserve your timestamps (and other metadata you may have lost).

I used rsync ver 3.2.4 for this answer, installed from MacPorts. I can't guess what features your version of rsync includes. You should consider upgrading rsync with MacPorts or Homebrew. Apple hasn't upgraded "their" version of rsync in many years, and it lacks the ability to support the methods and options used here.

Background: I began using rsync as my primary backup tool for macOS some time ago because I had to address the same issues as you: mangled and missing metadata. In my case, the issue was perhaps exacerbated by crossing a file system boundary - from APFS on my local drive to btrfs via SMB on a 'Synology NAS'. In your case, using Migration Assistant, I cannot imagine how Apple could not get the timestamp metadata correct - but you're not the only one who's reported this issue.

I made some notes while working to resolve my issue, and they morphed into a "recipe" in this GitHub repo. I'll keep this answer brief as most of the details are covered in that document.


Q: Can I use rsync to fix the timestamps that Migration Assistant trashed?

A: Yes - as shown below and verified

rsync with metadata preservation

rsync effectively copies data from a source (the "from" location) to a destination (the "to" location). If you are using a fileserver (NAS) as a backup for your Mac, the GitHub recipe will provide complete guidance to make and restore an rsync backup without losing metadata (attrs & xattrs).

However, as you need only to "migrate" files & folders from an old Mac to a new Mac, there is no need to involve an intermediate NAS. Consequently, the GitHub recipe can be simplified.

The scenario below assumes that the remote folders have been mounted (perhaps via 'File Sharing' in 'System Settings'). Alternatively, if you prefer to enable 'Remote Login' you can run rsync via SSH.

The following command code block shows the command syntax & options needed for rsync to perform an old Mac-to-new Mac "migration":

SCENARIO: from "oldMac" to "newMac"

Assuming the source & destination folders are:
SOURCE: /Users/MyHome/oldMacFolder/
DESTINATION: /Users/MyHome/newMacFolder

From Terminal.app on your "old Mac":

% SRC-FLDR="/Users/MyHome/oldMacFolder/"
% DST-FLDR="/Users/MyHome/newMacFolder"
% rsync -rlAXtgoDivv --dry-run -M--fake-super $SRC-FLDR $DST-FLDR

Notes:

  1. --dry-run this option prevents rsync from actually copying any files! It's used so that you can see what will be copied once the option is removed.

  2. If you've not set up file-sharing to mount the remote folder(s), you may use an SSH connection by adding the option -e ssh to the rsync command. This will require that you've set up 'Remote Login' in 'System Settings'.

Verifying Results - Preservation of All Metadata:

The GitHub recipe includes a short zsh script that will stat all of the files, folders, links, etc between the SOURCE and DESTINATION folders. It may be run after the rsync operations to verify that the chosen stat attributes have been preserved.

The rsync options presented here will do more than preserve the metadata containing the date-time stamps; they will also preserve all extended attributes (xattrs) . The GitHub recipe also includes a simple script to compare the output of xattr -lrsvx for all files (that have xattrs) in the SOURCE and DESTINATION directories.

The stat utility that I use here is the GNU coreutils version 9.1. The find version I use is the GNU findutils version 4.9.0. Both are readily available through MacPorts (and likely through Homebrew) for virtually every version of macOS. The xattr command is a macOS utility.


References and Further Reading:

  1. There’s more to files than data: Extended Attributes

  2. From quarantine to provenance: extended attributes

  3. From quarantine to provenance: how xattrs are copied

  4. Extended attributes

  5. How to use rsync over ssh

a few words re extended attributes
Source Link
Seamus
  • 5.1k
  • 3
  • 23
  • 44

Caveats: I'm not entirely sure this will resolve your issue or answer your question, but I hope it will help. I don't actually know what Migration Assistant does because in 12 years and 5 different Macs, I have never used it. So this answer may be a partial answer, or it may be incompatible with Migration Assistant. As a guess, I'd say use rsync after Migration Assistant - to clean up the mess it has left. In other words, rsync can likely restore not only the timestamps lost by MA, it can also restore all file metadata that may have been lost. Others here may have better ideas; the rest of my answer will be limited to rsync.

TheI used rsync version I used was3.2.4 for this answer, installed from MacPorts; version 3.2.4. I do not know if the rsync version included with your version of macOS includes the required options or notshown here - you may wish to investigate that instead of installing a current version of rsync.

I made some notes while working to resolve my issue, and they morphed into a procedure"recipe" on my macOS GitHub repo. I'll try to keep this answer rather brief since most of the details are covered in that document:.

rsync effectively copies data from a source (the "from" location) to a destination (the "to" location). If you are using a fileserver (NAS) as an intermediary between your two Macs, these commands mayshould do what you need. I don't see any need to involve an intermediate NAS, but it may be convenient or desirable if you want a backup. You should be able to accomplish the rsync operations over an SSH connection; the following commands illustrate the syntax & options needed regardless of whether you use the intermediary NAS or not:

from "old Mac" to "NAS"NAS:

Explanation: This rsync command uses the --dry-run option, and therefore will not actually move or modify any files. It will give you a detailed log (the ivv options) of what files would have been moved or modifiedwould have been moved or modified. Once you are happy with the results, simply remove the --dry-run option, and run again for effect. Review the rsync.log file using the "Decoding Table"

If you can make an SSH connection between your "old Mac" & "new Mac", I see no reasonit's not necessary to involve an intermediate NAS. The same rsync options may be used; I'll leave it to you to formulate the SOURCE and DESTINATION folders.

In theThe GitHub recipe is includedincludes a short zsh script that will stat all of the files, folders, links, etc between the SOURCE and DESTINATION folders. It may be run following rsync operations to verify that at least the chosen stat attributes are the same.

The rsync options presented here will do more than preserve the metadata containing the date-time stamps; they will also preserve all extended attributes. The GitHub recipe also includes simple script to compare the output of xattr -lrsvx for all files (that have xattrs) in the SOURCE and DESTINATION directories.

The stat utility that I use here is the GNU coreutils version 9.1. The find version I use is the GNU findutils version 4.9.0. Both are readily available through MacPorts, and available for virtually every version of macOS. The xattr command is a macOS utility.

Caveats: I'm not entirely sure this will resolve your issue or answer your question, but I hope it will help. I don't actually know what Migration Assistant does because in 12 years and 5 different Macs, I have never used it. So this answer may be a partial answer, or it may be incompatible with Migration Assistant. As a guess, I'd say use rsync after Migration Assistant - to clean up the mess it has left. Others here may have better ideas; the rest of my answer will be limited to rsync.

The rsync version I used was installed from MacPorts; version 3.2.4. I do not know if the rsync version included with your version of macOS includes the required options or not - you may wish to investigate that instead of installing a current version of rsync.

I made some notes while working to resolve my issue, and they morphed into a procedure on my macOS GitHub repo. I'll try to keep this answer rather brief since most of the details are covered in that document:

rsync effectively copies data from a source (the "from" location) to a destination. If you are using a fileserver (NAS) as an intermediary between your two Macs, these commands may do what you need:

from "old Mac" to "NAS":

Explanation: This rsync command uses the --dry-run option, and therefore will not actually move or modify any files. It will give you a detailed log (the ivv options) of what files would have been moved or modified. Once you are happy with the results, simply remove the --dry-run option, and run again. Review the rsync.log file using the "Decoding Table"

If you can make an SSH connection between your "old Mac" & "new Mac", I see no reason to involve an intermediate NAS. The same rsync options may be used; I'll leave it to you to formulate the SOURCE and DESTINATION folders.

In the GitHub recipe is included a short zsh script that will stat all of the files, folders, links, etc between the SOURCE and DESTINATION folders. It may be run following rsync operations to verify that at least the stat attributes are the same.

Caveats: I'm not entirely sure this will resolve your issue or answer your question, but I hope it will help. I don't actually know what Migration Assistant does because in 12 years and 5 different Macs, I have never used it. So this answer may be a partial answer, or it may be incompatible with Migration Assistant. As a guess, I'd say use rsync after Migration Assistant - to clean up the mess it has left. In other words, rsync can likely restore not only the timestamps lost by MA, it can also restore all file metadata that may have been lost. Others here may have better ideas; my answer will be limited to rsync.

I used rsync version 3.2.4 for this answer, installed from MacPorts. I do not know if the rsync version included with your version of macOS includes the required options shown here - you may wish to investigate that instead of installing a current version of rsync.

I made some notes while working to resolve my issue, and they morphed into a "recipe" on my macOS GitHub repo. I'll try to keep this answer rather brief since most of the details are covered in that document.

rsync effectively copies data from a source (the "from" location) to a destination (the "to" location). If you are using a fileserver (NAS) as an intermediary between your two Macs, these commands should do what you need. I don't see any need to involve an intermediate NAS, but it may be convenient or desirable if you want a backup. You should be able to accomplish the rsync operations over an SSH connection; the following commands illustrate the syntax & options needed regardless of whether you use the intermediary NAS or not:

from "old Mac" to NAS:

Explanation: This rsync command uses the --dry-run option, and therefore will not actually move or modify any files. It will give you a detailed log (the ivv options) of what files would have been moved or modified. Once you are happy with the results, simply remove the --dry-run option, and run again for effect. Review the rsync.log file using the "Decoding Table"

If you can make an SSH connection between your "old Mac" & "new Mac", it's not necessary to involve an intermediate NAS. The same rsync options may be used; I'll leave it to you to formulate the SOURCE and DESTINATION folders.

The GitHub recipe includes a short zsh script that will stat all of the files, folders, links, etc between the SOURCE and DESTINATION folders. It may be run following rsync operations to verify that at least the chosen stat attributes are the same.

The rsync options presented here will do more than preserve the metadata containing the date-time stamps; they will also preserve all extended attributes. The GitHub recipe also includes simple script to compare the output of xattr -lrsvx for all files (that have xattrs) in the SOURCE and DESTINATION directories.

The stat utility that I use here is the GNU coreutils version 9.1. The find version I use is the GNU findutils version 4.9.0. Both are readily available through MacPorts, and available for virtually every version of macOS. The xattr command is a macOS utility.

Source Link
Seamus
  • 5.1k
  • 3
  • 23
  • 44
Loading