5

Running OS X 10.10.2 on a Early 2009 Mac Pro. System installed on a 256 GB solid state drive. 10 GB of memory.

I'm having a problem with Time Machine where it's not backing up some of my files and folders. There doesn't seem to be much rhyme or reason to it, but it includes very, very important things like the parent directory for projects I'm working on, arbitrary hash directories in some git repositories, an assortment of emails, Messages attachments, random preferences files, and somehow the entire contents of my Pictures folder. And probably more -- I haven't checked everything.

I've recently forced Time Machine to do a full traversal when it made a recent backup (and confirmed that it was a full traversal by looking at backupd's output in Console.app), but these files are still missing.

I also tried going into one of the directories that wasn't being backed up and running:

find . -exec touch {} \;

to make every file appear to be recently modified, and then told Time Machine to make a new backup, but it still skipped them.

Does anyone have any suggestions for what I can do to fix Time Machine? I just want it to do a full backup, not skipping anything.

As a last ditch effort I could blow away my Time Machine drive and start from scratch, but I really, really don't want to do that because it contains several months worth of backups, and I occasionally do need to go back and get older versions of files.

Edit:

I haven't found a solution to this particular issue, but I have decided to discontinue using Time Machine for backups as I can no longer rely on it. I've decided to give ChronoSync a try since it seems pretty comparable.

Edit 2:

While re-reading some of my old questions I decided I'd make another update because I've learned more about this issue since my previous edit. I discovered that the cause of my data not being backed up was a bug in Qt's development tools, as detailed here. The fault was not Time Machine's. Since posting this question I've been using Time Machine again with no issues, though I still occasionally use my check-time-machine script to make sure everything is in fact being backed up properly.

4

I have the same issue since one of the mavericks updates, not necessarily the .0 version, not sure when exactly it started. I have this issue mostly with one folder: ~/Library/Preferences. I can fix the issue (see below), but 1-3 weeks later the issue is happening again with that folder. I think this folder might be hit often by the bug, cause the prefs-files in there are often changed, I dont know. Anway its a bug! If I should guess, the reason might be a bug in fsevents-daemon (fseventsd), which is responsible for tracking file system changes.

Sometimes I also have other folders than Preferences not backing up anymore, but this happens not that often.

To test which folders are "stuck" and do not backup properly you can:

  1. do a fresh backup and keep the backup volume connected
  2. in Terminal: tmutil compare -n

The terminal command runs 30 mins with my backup of 150 GB size. It helps identifying files which are not in the Latest backup. Be informed, some files might be shown, which are ok to be not backed up.

I also tried touch files, which did not help. What helped, was renaming the parent folder. To fix the issue for a particular folder, e. g. ~/Library/Preferences, I rename the folder twice like:

in Terminal:

sudo mv ~/Library/Preferences ~/Library/PreferenceS
sudo mv ~/Library/PreferenceS ~/Library/Preferences

Note, the last letter s/S at the end. I need sudo for the Preferences folder, cause OSX has set special access rights for that folder. So sudo might not be needed for other folders. After doing so, the files in that folder are backed up correctly again. Problem: the issue is not solved permanently. Unpredictably after 1-3 weeks the issue is back.

other links:

  • Thanks for the suggestion. After running tmutil compare -n I can see that there's a LOT of files not being backed up scattered all over my hard drive. I could try to rename some of the folders but it'd be a huge pain, and some of them are important system folders like /Library. I think this is the final nail in the coffin for Time Machine -- I'll have to use something else for my local backups. Which is a pity, because Time Machine's interface is really nice. – GuyGizmo Feb 3 '15 at 17:16
  • tmutil compare -n shows every single file. A closer look might condense it to a few folders to be fixed. Sometimes its not the first level parent folder, sometimes you have to step up to parent-of-parent folders. Just start with folders that contain many files and fix them first. The next runs of tmutil compare -n will be more cleaned :/ – Pete Feb 3 '15 at 17:22
  • I did take that into account. For whatever reason the files not being backed up are truly scattered all over the place. In some cases there is a parent folder not being backed up, but in most cases it's scattered files throughout a lower level folder like ~/Library. I could get Time Machine to back them up by renaming the lower level folders, but frankly I want to use a backup option that I can actually rely on, so I'm just going to switch to something else. – GuyGizmo Feb 3 '15 at 21:03
  • Sry, I need to clarify, cause the details matter. To be clear, you have to look for folders which content is incomplete, not completely missing folders. When the bug appears e.g. 300 out of 600 prefs files are not backed up. Also subfolders in Preferences/ might completely be skipped. But the reason seems to be the all including folder Preferences/. I understand you do not trust TM now. I do a compare check once a month. But, I would like to know if you can fix some of your misses by renaming the mad parent-folder, to confirm we actually have the same issue? – Pete Feb 3 '15 at 21:56
  • Just tried renaming a couple of my folders that had missing files. No dice -- Time Machine still isn't backing them up. – GuyGizmo Feb 4 '15 at 5:58
2

Both I and my housemate ran into this problem on separate iMacs. It first cropped up around the time we upgraded to Mavericks. After battling it for weeks I can confirm that (a) messing with your preference files won't fix it and (b) starting over with a new Time Machine backup disk and clean preference files won't fix it. Time Machine may behave itself for a while but within one to four weeks it will start skipping files and folders at various levels, particularly in folders like ~/Library/Preferences that see a lot of change.

I'm chiming in in hopes of saving others some frustration. Run tmutil compare -n immediately after a backup and see if anything critical is missing. If so switch to an alternative backup system as soon as possible. Numerous folks have posted about this over the years and no one has ever gotten Time Machine working again once this has cropped up.

You may run across this thread which details similar problems in Mountain Lion. At this point that thread contains a number of red herrings related to corrupted Time Machine preference files which don't seem to be relevant for Mavericks or Yosemite.

2

I just recreated a Time Machine backup from a completely blank hard drive. I found that it was still skipping files, but after digging in more, I discovered that lots of my files (for reasons I can't fathom) were marked as being excluded from Time Machine. Specifically, the files were marked with sticky exclusion.

I wrote a script to scan my entire filesystems for things that are being excluded from Time Machine and report back, differentiating between which files are meant to be included but aren't and which files are meant to be excluded. It can then optionally remove the exclusion marker from all of the excluded files it finds. Once I did this, all of my data was backed up properly.

I am now running OS X 10.11.1, so I'm not sure if the files that were missing from when I originally made this thread were due to being excluded (though I can confirm at least a few of them were), or if there were bugs in Time Machine that's been fixed in the subsequent OS updates, or if clearing out my Time Machine backup and starting completely fresh fixed it. Will update this thread as I learn more.

Script can be downloaded here: https://github.com/briankendall/check-time-machine

  • Wow, I just found that my folder $USER/Library/Saved Application State has not been backed up anymore since 2016-11-19. I tried my old rename trick but it did not help. Then I checked with tmutil isexcluded "$USER/Library/Saved Application State" and indeed it was excluded. WTF! Thanks for your hint, I ll check now my entire filesytem with your script or something similar. – Pete Jan 15 '17 at 12:45
  • Btw, my Preferences folder still gets in the state of not beeing backed up correctly from time to time. I am still on latest Mavericks. And with that folder the rename trick still helps fixing the issue until it happens next time. I have scripted the detection of the error state and the repair process so I can live with the TM-bug somehow. My scripts only do that with the Preferences folder, since it was the main issue for me. Lets see what other folders got the excluded bit meanwhile. Also I have to run an entire scan with tmutil compare -n to check my backup. Its been I while. – Pete Jan 15 '17 at 12:55
  • Note: tmutil isexcluded "$USER/Library/Saved Application State must be run after cd /. Or you run cd $USER ; tmutil isexcluded "Library/Saved Application State". The tmutil always assumes relative paths. – Pete Jan 23 '17 at 6:32
  • I have removed the exclusion several times now from the folder Saved Application State. But it gets excluded again and again. I have no idea why and what todo about that. Any ideas? – Pete Feb 5 '17 at 10:00
  • @Pete I suspect that Apple has deliberately set it up so that Saved Application State gets the excluded bit set again and again, because they don't intend for it ever to get backed up. And it makes sense; much like a cache, the Saved Application State folder isn't actually vital and doesn't (or at least shouldn't) contain any of the important data of an application. Have you ever gotten it to back up at any point? If so, you could probably create a launchd job that fires at regular intervals and removes the exclusion bit if it's really important for you to back it up. – GuyGizmo Feb 5 '17 at 15:16
1

The Time Machine preference file seems to be corrupted.

  1. Open the Time Machine preferences and put the slider to OFF.
  2. Go to /Macintosh HD/ Library Preferences and drag to the Trash the file com.Apple.TimeMacine.plist and restart the machine.
  3. Open the Preferences, click Select Backup Disk and Use Disk.
  4. The backup will start and will take some time depending to the number of files.

You have than a complete backup that will be added to the existng backups.

1

The way I would fix this programmatically is to add an exclusion for the file and then remove it.

FILE=/the/path/to/whatever/file/you/wish.foo
tmutil addexclusion $FILE
tmutil removeexclusion $FILE
tmutil isexcluded $FILE

If the last result is still that the file is excluded, you might need to use the -p option which requires sudo or look at if the file or the path leading to it is excluded by rule.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .