22

Since updating to 10.7.5 both Spotlight and Time Machine are very slow.

  • To index a normal system drive Spotlight calculates to need several days.
  • As a result Time Machine takes up to an hour or more for an incremental backup of ~100 Mib.

What solution is there for these slow Time Machine backups and Spotlight indexing?

Improve this question
7
  • I can confirm both slowdown since 10.7.5. Where a fix for Time Machine slowness is most urgent. Spotlight indexing finished within a few days but Time Machine is running for days and only at 11,59 GB of 53,19 GB.
    – Pro Backup
    Commented Sep 27, 2012 at 10:41
  • But even a Spotlight index that takes days is uncacceptable.
    – user18805
    Commented Sep 27, 2012 at 10:42
  • Does Spotlight ever finish building the index or does it just shown the remaining time as multiple days?
    – Lri
    Commented Sep 27, 2012 at 12:56
  • @LauriRanta I didn't wait :)
    – user18805
    Commented Sep 27, 2012 at 12:57
  • 1
    The solution which I found worked was to delete /var/folders then reboot. The spotlight index time went from 3 days down to 51 minutes. Likewise Timemachine, on a newly installed/restored Lion, went from 30 days to about 11 hours.
    – Grant Sayer
    Commented Oct 3, 2012 at 11:18

11 Answers 11

Reset to default
5

The OS X Lion 10.7.5 Supplemental Update released October 04, 2012 includes:

Resolves an issue that may cause Time Machine backups to take a very long time to complete

and, after testing, this seems to have completely resolved the issue.

Note that for me, the first backup after installing this update still took unexpectedly long, and included many com.apple.backupd[423]: Waiting for index to be ready (100) messages, so maybe Spotlight hadn't yet properly re-indexed the drive. Now, a couple hours later, small backups complete at their usual brisk pace and all is well.

Improve this answer
4
  • 1
    FYI. This may have solved the problem with 10.7.5, but not with 10.8.2.
    – jschmidt
    Commented Oct 8, 2012 at 15:21
  • @jschmidt Did you try the 10.8.2 supplemental? The 10.7.5 supplemental update included changes to lsboxd, xpchelper, and a bunch of audio stuff. Also changes to xpchelper's SandBox permissions to include ocspd (though no changes to mdworker stuff). What does /System/Library/Sandbox/Profiles/com.apple.xpchelper.sb look like post supplemental?
    – Old Pro
    Commented Oct 17, 2012 at 21:39
  • Yes. I tried the supplemental update on 10.8.2, but it didn't help. The only thing that has worked for me is a complete system wipe and reinstall of the OS. I manually copied my data from another backup, but did not restore any apps or settings.
    – jschmidt
    Commented Nov 13, 2012 at 17:09
  • It's a long time since I updated, but I still get this problem from time to time. Any thoughts?
    – D A Vincent
    Commented Aug 4, 2016 at 3:06
10

I posted this workaround in that topic. Here it is (it is not a solution, only a temporary workaround to let my backup and Spotlight work until a fix is released).

Using Terminal:

  1. I disabled Spotlight indexing:

    sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

  2. I deleted Spotlight index in all my drives (including the backup drives):

    sudo rm -rf /.Spotlight-V100/Store-V1 /.Spotlight-V100/Store-V2  /.Spotlight-V100/VolumeConfiguration.plist

sudo rm -rf /TimeMachineDrive/.Spotlight-V100/Store-V1 /TimeMachineDrive/.Spotlight-V100/Store-V2  /TimeMachineDrive/.Spotlight-V100/VolumeConfiguration.plist

(rm -rf /.Spotlight-V100/* did not work for me.) (This can also be done by excluding the whole drive using privacy setting of Spotlight and then reincluding it.)

  1. I deleted /var/folders content (see this topic to know why):

    sudo rm -rf /var/folders/*

  2. I deleted the .inProgress files from my Backups.backupdb subfolders (put to trash then empty the trash is the only way to do this).

It may be necessary to reformat your backup drive instead of only deleting .inProgress files.

  1. I rebooted and then I restarted Spotlight:

    sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

sudo mdutil -i on -E /

  2. I waited until Spotlight finished indexing process

  3. I launched a new backup using TimeMachine

  4. I opened a bugreport to Apple with the title: "TimeMachine hangs when Spotlight is in use under Mac OS X 10.7" praying that many other people will do the same to make this bug a priority.

Some steps may be useless, but I took all what I learn by reading different topic about slow TM and Spotlight.

Improve this answer
2
  • Thank you. sudo rm -rf /var/folders/* did the trick in my case
    – maggix
    Commented Oct 4, 2012 at 20:08
  • Do not remove the content of /var/folders on El Capitan, it will totally hose your system: the progress bar would stop during the log in, I had to reinstall a TM backup. For the record, TM was impossibly slow on my machine due to MenuMeters.
    – Frank Pavageau
    Commented Sep 1, 2016 at 14:31
7

Check your console log file and look for lines like this (the real log lines will have process IDs and dates):

sandboxd: mdworker deny mach-lookup com.apple.ls.boxd
mdworker: Unable to talk to lsboxd

If you're seeing that, it is probably the source of the problem. Something Appple did with this latest release (also affects Mountain Lion 10.8.2).

I don't know what's really going on other than to say that mdworker (which scans and indexes files for Spotlight) calls LSCopyLibraryItemURLs() which eventually sends a message to lsboxd. The Sandbox policy doesn't allow that, so the worker is killed.

So maybe the policy file is wrong, or maybe mdworker shouldn't be making that call. I don't know. However, fixing the policy file is relatively easy. POTENTIALLY DANGEROUS, but easy.

Edit these files as root:

/usr/share/sandbox/mdworker.sb
/usr/share/sandbox/mdworker-bundle.sb

Inside the files you'll find the line:

(allow mach-lookup (global-name "com.apple.ocspd"))

Right below it, add the line:

(allow mach-lookup (global-name "com.apple.ls.boxd"))

That's it!

If you want extra comfort, maybe reboot. Maybe erase your spotlight index and start over:

sudo mdutil -E /

In fact, I still see mdworker: Unable to talk to lsboxd in my log files but at least now the indexing will finish and Time Machine moves quickly again.

Improve this answer
8
  • There are three files: mdworker-lsb.sb, mdworker-scan.sb, and mdworker.sb. They all contain only the line (allow mach-lookup) but not (allow mach-lookup (global-name "com.apple.ocspd")).
    – user18805
    Commented Sep 29, 2012 at 9:52
  • @OldPro I tried to follow your instruction, and am in the same situation as Tichodroma. So now in mdworker.sb i have (allow mach-lookup) (allow mach-lookup (global-name "com.apple.ls.boxd")) and after I added that line I restarted. Yet time machine is still going snail slow. Can you help please?
    – rabbid
    Commented Oct 2, 2012 at 0:25
  • I confirmed that I had those messages in the console log. I followed the instructions as well. No change for me. I deleted the entire TM bundle for my machine and started a new backup. It made it about 200GB through a 600GB backup and stops. I'm running TM on a Mac Mini Server with a Drobo connected via FW800. My source machine is a MacBook Pro Retina running ML 10.8.2. I rebooted both the server and MBP and tried to continue the backup again. This time, it only made it through about 100GB more and then stops. One more try only did 3GB. Something is stopping it, but I don't know what.
    – jschmidt
    Commented Oct 2, 2012 at 17:03
  • 1
    @Tichodroma I'm so sorry I misread your comment. If your mdworker.sb already had (allow mach-lookup) then (1) you should never bee seeing sandboxd: mdworker deny mach-lookup com.apple.ls.boxd in the logs and (2) editing the file won't help. That goes for you, too @user15380
    – Old Pro
    Commented Oct 2, 2012 at 21:44
  • Okay. I found another commonality (seemingly) to the backup failures. I no longer have the deny mach-lookup com.apple.ls.boxd in the console (although I still get the "mdworker: Unable to talk to lxboxd"). However, I do see a bunch of "sandboxd[1289]: ([1288]) mdworker32(1288) deny mach-lookup com.apple.PowerManagement.control ..." messages. Do I need to add another line for com.apple.PowerManagement.control?
    – jschmidt
    Commented Oct 3, 2012 at 20:36
6

You can completely fix all the mdworker related sandbox errors in Console (including the cannot talk to lsboxd) by editing the system.sb profile rather than the mdworker ones. The system.sb file is used by all Apple system processes that run under the sandbox, including mdworker. If you modify this file (which is located in /System/Library/Sandbox/Profiles) to include:

(allow mach-lookup (global-name "com.apple.ls.boxd"))

(allow mach-lookup (local-name "com.apple.ls.boxd"))

This will stop all the lsbox related mdworker issues.

Improve this answer
1
  • 1
    But, the comment at the top of system.sb says, "The contents of this file are also auto-generated and not user editable; it may be overwritten at any time."
    – Jeff
    Commented Jan 24, 2013 at 6:10
5

If Spotlight never finishes building the index, it can be because the indexing always stops on certain files. I made a clean installation a few years ago because of that issue, but it returned after I restored my old files. I ended up narrowing down the files that caused it by just adding folders to the privacy tab in System Preferences.

Someone at Super User ran opensnoop to find what files mdworker stopped on: 

sudo opensnoop -n mdworker

From a suggested edit by user37651:

I ran the opensnoop command above and it showed a problem with files in ~/Library/Developer that were put there from a previous Time Machine restore. (I currently don't have XCode installed.) I deleted the folder, and Spotlight indexing finished in only 30 minutes. My Time Machine backup now seems to be working.

Improve this answer
2
  • 1
    Maybe I'll try this later. The strange thing is that Spotlight worked fine in 10.7.4 and only slowed down after the update to 10.7.5. The files Spotlight has to index didn't change.
    – user18805
    Commented Sep 27, 2012 at 13:05
  • I've tried your approach by adding everything on the system drive to the privacy list of Spotlight. But even then Spotlight took so long to calculate how much time it will need to index the files (which?) that I've aborted the process. Apple, where is the fix?!
    – user18805
    Commented Sep 27, 2012 at 15:46
1

It is possible to delete the Spotlight index and then force Mac OS X to create a fresh Index from scratch. This is usually worth trying in a situation like yours.

I'm afraid I don't know the Terminal commands for doing this. When I need to do it, I use the commercial program MacPilot, from Koingo Software, which costs US $20. They have a fully-functional trial version that you can use for 15 days. If you find it useful, you can decide to pay for it.

I suppose there are other utilities for doing this also.

Of course, once you delete the existing Spotlight index, you need to wait for Spotlight to create a new index. You should let your Mac run uninterrupted for as long as it takes to generate a new one completely.

enter image description here

Improve this answer
3
  • The problem is that Spotlight calculates the time needed for the index to be many days. So using MacPilot or the mdutil command line tool helps to ease the pain but it doesn't solve the problem.
    – user18805
    Commented Sep 29, 2012 at 9:56
  • Do you think that the disk drive itself may be corrupted or physically damaged, and that is why the indexing is projected to take so long? Perhaps you should run a diagnostic for disk damage. Clone your system to an external backup disk while you are at it to be safe.
    – user9290
    Commented Sep 29, 2012 at 19:08
  • No, the drive is fine. Besindes, many other users have the same problem. Take a look at the discussions on Apples site. I have external backup using CCC :)
    – user18805
    Commented Sep 30, 2012 at 5:40
1

Damn! Got faster boot times too

I am writing a different answer just to summarize the specific steps I took (I must keep clear that they are part of what is ALREADY written here, not my contribution), because after I took those steps, not only Time Machine is working faster, but my boot time decreased from 1min10secs to 40secs! My boot time always intrigued me as I constantly saw people reporting much lower boot time for the same specs and, somehow, those steps cut 30 seconds off my boot time (Yeah, I tested more than once and, yeah, caches were ok and I took many steps before taking those ones here, including repairing permissions, repairing disk, even defragging with iDefrag2, all to no results). Until now :P

1

sudo mkdir ~/trash_backup
sudo mv /var/folders ~/trash_backup/ # backup just in case

2

After that, editing /usr/share/sandbox/mdworker.sb and /usr/share/sandbox/mdworker-bundle.sb, adding (allow mach-lookup (global-name "com.apple.ls.boxd")) below the line (allow mach-lookup (global-name "com.apple.ocspd")) on both files.

3

Finally,

sudo mdutil -i on -E /; sudo reboot

Thanks lauhub and Old Pro. Can anyone explain the decrease on boot time and what the mach-lookup on boxd thing does and possible security issues? Thanks all :)

Improve this answer
1

So I've suffered the same as everyone else but unfortunately didn't read all the multitude of threads in the correct order so basically lost my original TM backup, then my Spotlight db in a failed effort to fix it.

A thread somewhere which I can't find again (I think it was more about slow Spotlight index) had a post from a guy who simply reinstalled 10.7.5 and everything was back to normal. So I did that, and it worked.

After completion of the reinstall I left TM off and allowed Spotlight to index. I went out for a ride and when I came back it was done - 149GB in something less than 90 min. And it works, I've searched for stuff on my HD.

I then turned TM on and it took 2hrs 9min to complete.

So if you are considering spending the time and bandwidth downgrading to 10.7.4 you may wish to consider a reinstall of 10.7.5.

Improve this answer
1

All, I had the same issue, TM anticipated to take days for a brand new backup on a new disk, and indeed it took hours for some 200MB to be completed.

I disabled spotlight with the command indicated above, and - behold - 40GB loaded within 10min and the TM updated is running as smoothly as one would expect.

This is quite an important bug and its disappointing that Apple sends out an "update" that causes problems like this.

Improve this answer
0

According to https://discussions.apple.com/thread/4324046?start=60&tstart=0 disabling Spotlight with $ sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist improves the Time Machine backup speed.

Improve this answer
3
  • This may or may not work. But since Time Machine relies on Spotlight to know which files need to backup up, this is no real solution. Besides, Spotlight is an essential part of OS X.
    – user18805
    Commented Sep 27, 2012 at 11:43
  • 1
    Indeed this is only a workaround not a solution. In my case it doesn't even speed things up much.
    – Pro Backup
    Commented Sep 27, 2012 at 11:49
  • 1
    This did not work for me. In my case, TM hangs after a time. Disabling Spotlight did not help at all.
    – jschmidt
    Commented Oct 14, 2012 at 13:53
0

A simple rebooting into SAFEMODE and then rebooting back to NORMAL MODE appears to have cleared this use up for me. Went from 3 weeks to 1 hour for back up.

Improve this answer

You must log in to answer this question.