I have a late 2011 Macbook Pro, running Mavericks 10.9.2. Its sole HDD is a 750GB drive, formatted with Bootcamp. It's still running reasonably well, but in running a defragment pass on it, I've identified that there are a bunch of files which are refusing to be moved by the defragmenter (iDefrag).

iDefrag reports a POSIX error code of 5 when accessing the files. Picking one at random and trying to copy the file to another location in the shell also reports an error, which makes me think the problem is real and with the disk / FS. Output of cp is:

cp: unity_nophysx.nexe: Input/output error

Error code 5 is 'access denied' as far as I'm aware, but the defrag process is running as administrator and running cp using sudo on the suspect file makes no difference.

Disk Utility, fsck and the Apple Hardware Test all claim the disk is fine. No SMART errors reported, and while there were some permissions errors, they weren't with the files iDefrag is complaining about, and Disk Utility claims to have fixed them without complaint.

There are maybe a hundred or more corrupted files, but still a very small fraction of the drive. As far as I can tell, no system files or crucial data are affected. While it would be nice to retrieve the data, I don't mind reinstalling or going to backups. At this point I don't know if it's really the drive dying, just some bad sectors due to the drive being moved while writing, or some other minor corruption that can be worked around. I'm assuming the worst case, and that most likely I'll have to get a slightly larger HDD and clone the existing drive to avoid having to rebuild the system.

My question is really how I go about marking those broken files as properly broken and fixing or purging them, so that a clone of the disk will succeed and not get hung up on files / blocks it can't access. Disk Utility isn't seeing the problem, and I don't know of any command line or third party tools which will do the job. I don't want to write off the entire disk and start from scratch, as the drive seems otherwise healthy, so I'm looking for repair / diagnostic tools.

up vote 6 down vote accepted
+50

If you are facing a healthy file system at the level of its structure and want to find files which have disk faulty blocks, here is how I would proceed:

  1. Make a full backup of your disk with Time Machine or Carbon Copy Cloner

    Check this backup.

  2. Run the following heavy and risky (in case you do have bad blocks outside of your filesystem structure) command (make sure the {} is quoted so filenames containing spaces work):

    find / -type f -print -exec dd if="{}" of=/dev/null bs=1m \;
    

This heavy find command will print for any plain file its name (thus not reading it, but just its directory entry) and then continue making a full and fast read of all its data blocks.

Upon hiting the first file containing bad blocks, this find will cause the kernel to log read error on /var/log/system.log, and it will either slow down or bring your system to a total halt. This will mostly depend on the hard drive capacity to relocate the bad blocks found on its internal pool dedicated to this usual fix task. This file containing bad blocks will be the last name printed by find.

Write down this file name on a piece of paper! Let's say that this file name is:

/.DocumentRevisions-V100/.cs/ChunkStorage/0/0/0/9

At this point you may have the possibility to kill find quickly by hiting ctrl+C. If killing it nicely is failing, just crash your Mac.

Upon rebooting your Mac, directly check the file containing bad blocks:

dd if='/.DocumentRevisions-V100/.cs/ChunkStorage/0/0/0/9' of=/dev/null bs=1m

If the command terminate correctly, then the error was light enough for your disk to be able to read this file and reallocate the bad blocks.

  • If the command doesn't terminate, you won't be able to kill it normally, your data is totally lost, and you will have to crash your Mac once more.

In this last case, you have to consider replacing your disk and to work from your last backups. Some other files might also contain bad blocks and may have stayed undetected since a long time as long as you didn't read them.

The kernel won't fire a read error on a block you never read.

  • Aha, this is absolutely the sort of trick I was hoping for. First pass with the find/dd script touches all the files/blocks on the disk, and sure enough I find a bunch of files which give "Input/output error", and I can simply output the log of the command to a file and then grep it to find out which files are duff. It seems like the dd command isn't itself enough to trigger any sort of automatic fixing (I didn't even know OS X did that), but at least it gives me a reliable way to identify the files. – MrCranky May 10 '14 at 20:26
  • On the plus side, when the OS tries to read from the files with these bad blocks, it doesn't crash or hang up horribly. I see a May 10 20:42:15 ICE kernel[0]: disk0s2: I/O error. pop up in the logs, but no clue as to which file triggered it. But then the command runs on quite happily. – MrCranky May 10 '14 at 20:28
  • Your kernel doesn't hang with the BBFH because your disk still have enough blocks available within its pool to fix bad blocks. dd doesn't fix anything, this command purpose is to copy data and convert them as fast as possible. The disk is still able to repair light errors. Stay watchful, the price of a disk is nothing compaired to your work. – daniel Azuelos May 10 '14 at 21:10
  • Mmm, yes, I'd assumed that: dd just being a dumb tool to ream all the data out of one file and put it somewhere else (in our case, into thin air). What really matters is that every block associated with the file is read. What I'm not getting is what you expect OS X to do in that case. Clearly the kernel can't read these bad blocks, but do you think the disk itself can and may fix them? If it can't get the data out of the original bad block, how is it going to shift it elsewhere? – MrCranky May 10 '14 at 21:21
  • Excellent question. The disk will automatically make retries on reading blocks. Everytime the head position is mechanically in a different position. If one of this attempt is succesfull, the data is copied on one of the block available to repair bad blocks. The bad block is flagged as bad and will never be used again. On the other hand, if all the retries fail, then the data isn't saved, and after a very long time, the disk will flag the block as bad, and allocate a new empty one to the visible disk. The kernel will report an unrecoverable disk error. – daniel Azuelos May 10 '14 at 21:29

Reboot in single user mode by holding the Command + S during boot. When you see a prompt (should look like root # or something similar), type fsck -f and press Return. This is Mac's built-in filesystem consistency check tool and allows you to find and repair errors with the startup file system. Run this command until you don't see **The volume [volume name] was modified.** or the tool fails three times in a row.

If the tool fails, it might be indicative of a larger problem (but I couldn't tell you what without seeing the output of the tool). In any case, make sure you've backed up everything you can before running any disk tool. When you've finished, type reboot into the prompt and press enter to (you guessed it!) reboot your computer.

For extra information you can find the fsck manual pages here.

  • Interesting, but it looks very much like fsck, even with -f and in single-user mode, is doing just what Disk Utility did. Like Disk Utility, it finds nothing and thinks the disk is just fine. I'm presuming that it is scanning the file-system records, but I think my problem is at the block level - i.e. the file system is well structured, but the actual data within the files can't be accessed when it comes to reading/copying/defragmenting them. – MrCranky Apr 29 '14 at 7:37
  • 1
    → MrCranky: right! fsck & Disk Utility are checking the file-system structure integrity. They read the disk blocks allocated to the file-system structure. They are not made to verify data blocks integrity. Hence they can run on a disk with failing blocks without rising any read error. If you want to check your disk, even blocks which may be faulty but are actually unused, just use a basic tool as dd if=/dev/disk0 of=/dev/null ibs=1k and within another shell window run tail -f /var/log/system.log. This is free, extreme and won't hide you any error. – daniel Azuelos May 10 '14 at 10:06

I would highly recommend DiskWarrior for rebuilding disk catalogs and for scanning for potentially damaged files.

During the catalog rebuild, it can also let you know if it experiences a delay because of disk malfunction.

  • I'm not averse to buying a tool to help, but with no trial, and no guarantee that it is even designed for finding the sort of errors I'm experiencing, I'd need a lot more recommendations to back up yours before being prepared to drop $100 on a tool. – MrCranky Apr 29 '14 at 7:45
  • -1 Not just an answer, but a mix of comment and answer. – Max Ried May 9 '14 at 8:08

As you say, it is not even clear those files are damaged, at least your Mac does not thinks so.

Every OS makes unmovable files that are needed for its operations (Restore points, currently active files ect....). Some defrags will show them, some will not.

The fact that you can not access or move them does not mean they are damaged.

Normally Mac's are very good in taking care of them self.

Using Apple maintenance is done by: open the Terminal and type:

sudo periodic daily weekly monthly 

followed by Return, enter your Administrator password, and OS X will take care of things for you.

Look in the Console for the reports on those if you are interested.

While in the Console look (search) for any I/O errors that would indicate your disk is starting to have problems, to compliment the Disk Utility and the fsck findings.

On occasion I use a free tool called OnyX for additional maintenance task. It is made by French and as they food it is just great :)

OnyX is a multifunction utility for OS X which allows you to verify the startup disk and the structure of its system files, to run miscellaneous tasks of system maintenance, to configure some hidden parameters of the Finder, Dock, QuickTime, Safari, Mail, iTunes, login window, Spotlight, and many of Apple’s applications, to delete caches, to remove a certain number of files and folders that may become cumbersome, and more.

With all this said, I am not questioning your decision for using the defragmenter (iDefrag) since I do not know it, but rather offering alternative solutions.

  • Use of the defragmenter is not the issue, I'm perfectly aware of what OS X does and doesn't do in that regard. The files were definitely not in use, these were data files for an application which was not active, and indeed the application now cannot be moved. – MrCranky May 9 '14 at 7:39
  • On Onyx - it again is doing little more than Disk Utility does - checking the SMART status of the disk and then running the fsck style diagnostic (which as we've established thinks there's nothing wrong) – MrCranky May 9 '14 at 8:27
  • Just to be clear, for anyone else reading this answer, the files most definitely were damaged, and the Mac knew that, because I wasn't allowed to read from them (copy them, whatever). That wasn't because they were system files, or in use at the time, it was true even for user data files. Periodic maintenance did not help with the problem, again because like fsck it appears to only care about file system issues, not block accessibility problems. The Console showed errors only when I manually tried to copy/read the data from one of these broken files, it was no help in finding them. – MrCranky May 12 '14 at 7:48

Working off Buscar's answer, you can do this automatically using some pretty heavy command line foo.

sudo find / -type f -print0  | xargs -0 -I{} dd if='{}' of=/dev/null bs=1m 2>&1 | grep 'error' >>badfiles.txt  & 
  • sudo:Admin mode
  • find -print0: absolute path
  • xargs -0 -I{} : substitute {} in the next command
  • dd 2>&1: redirect std error to stdout
  • pipe stdout to grep looking for string error
  • Append results to a list file.( note: This should be on external media if you believe your internal drive is iffy)

As unreasonable as it sounds, before doing anything you should duplicate all your data to a known good drive. If booting from the installer and copying the data fails, there is a command line utility called 'dd' that can do low-level duplicating and in a far more uncompromising way.

 man dd

for more information on dd including use and proper syntax.


Another vote for Matt's post, boot single-user mode, and run

 fsck -fy 

over and over until fsck stops reporting errors.


A vote for Adam's post, DiskWarrior is a simple to use but very powerful application that will report HDD failures, check individual files for errors and repair them if possible, and rebuild and optimize directory structures.


Another possible solution that may sound unreasonable but is often a last ditch attempt to recover data with a lot of annacdotal evidence for success is to pull the drive, protect it from moisture using a couple layers of freezer bags, and place it in your freezer for 30-45 minutes. Then while the drive is cold, mount the drive in an external usb dock, and use another temporary system to again attempt to copy the corrupted data to another drive. Generally, this is used if there is a hardware problem and the drive is failing. If you can duplicate the entire drive with your data intact, this is ideal, as often a repartition and reformat will give the drive a new lease on life.

  • As I've said, fsck does not report any errors. The disk is not yet temperamental or reporting random errors, and the list of files which are corrupted does not seem to be growing, so I don't believe I'm anywhere near the 'freeze for a last emergency pull' stage yet. I'm also already very well backed up at a file/folder level, and not worried about losing data, as I said in the question. Good to hear another vote for DiskWarrior though. – MrCranky May 9 '14 at 7:42
  • @MrCranky : I believe you refer to something posted prior to your updating your question; I was reinforcing the fsck idea for any that find this page seeking a solution to similar symptoms. In regard to anything I posted about HDD failure, it never hurts to be comprehensive, again, for others and not necessarily you personally. I've seen my fair share of hard drive failures. Often there is no indication of failure, even with SMART tech, until you can no longer access the data by any means. If you care about the data, I strongly recommned you get a new drive, and backup your data. – chillin May 9 '14 at 20:50
  • I certainly don't disagree with the recommendation for backup, but the spirit of the Q&A format is to answer the question that's posed, not a generic "how do I fix a broken disk" question (of which there are many). Well before I edited it to add fsck to the list of "things which think the disk is fine", I'd replied to the answer mention fsck discounting its usefulness. fsck and Disk Utility perform much the same function, and that is to operate on the file system structures, not at the block level. I did try to be pretty specific that this is a block problem, not a file system problem. – MrCranky May 12 '14 at 7:44

For a single file that cannot be read in its entirety due to a disk read error, you can use the dd utility to duplicate the file to an external volume, substituting NUL bytes for the blocks that cannot be read. It is highly recommended to duplicate to a different volume (e.g. "USB Disk" in the example below).

Example:

dd if=/path/to/damaged/file of=/Volumes/USB\ Disk/file bs=512 conv=noerror,sync

By using 512-byte blocks, the maximum number of readable blocks will be recovered.

Recovery may take a long time, as the kernel will block for some time on each failed read.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.