0

I have an existing 1T USB drive (disk5 in the screenshot). I formatted another 1T USB drive (disk4) and starts to rsync files from disk5 to disk4.

Near the end (the current state in the screenshot) I still have 100+G of files that I'm unable to copy to disk4.

From the screenshot, there are still about 250K files and 26 folders not copied over to disk4. Yet disk4 has less free disk space.

Based on finder, the size of the uncopied files is 110G.

Things I that tried:

  1. empty Trash
  2. run rsync with --delete

disk4 is essentially a "rsync -a" backup of disk5, what cause the huge discrepancy in disk space?

Anything else I can do or missed?

Disk Utility Information for both drives

UPDATE

Based on hunch, I did the following.

  1. reformatted disk4 again
  2. using Finder, first copy some folders with huge files (VM images)
  3. after the above copy completed, copy rest of folders

All files fits into disk4 this time!

I'm not sure, which of these fix it.

  1. reformatting
  2. copying large files first
  3. not using rsync
  4. none of the above

UPDATE 2

Here's the Information for both drives after the above Update.

Notice that:

  • Used space is almost the same (0.18G less in disk4)
  • 36203 files more reported in disk5
  • 27 folders more reported in disk5

"rsync -av" did not report any more files to sync.

I guess Apple is hiding some files.

Disk Utility Information for both drives after reformatting disk4

  • Today I formatted (journaled,encrypted) a new usb 1T drive. Encountered the same issue during rsync files from another 1T drive to the new drive. From disk is 75G available, new disk ran out of space at 5G available. – kenchew Feb 23 '16 at 8:11
1

You could look to folders that you can zip into a compressed folder. This should give you additional space on disk4, so long as you erase the files that have been added to the compressed zip file. You can either use Compress Files in Finder or zip in Terminal.

UPDATE

While this does not address rsync directly, this is another option that you may consider.

If this is solely a backup, and you are not actively using the data on disk4, i.e. it would only be used to restore lost data, or recover incremental data, then I would say try CrashPlan by Code42. The software is free and you do not need to buy their premium cloud backup services to use the local backup functionality. The only thing is that without the premium plan, I do not think that they unlock the ability to set the backup schedule for local backup. They do however give you a thirty-day free trial for the premium service, so you can determine how important having control over the backup schedule is.

It works very well, and is very light on resource utilization, so you won't know it is there. I in fact view it as a flaw in the software, as they are only addressing a single core of multicore processors, which can extend initial back up times. You have control over how much you allow CrashPlan to use your systems resources.

The benefit is that it significantly compresses the data to the backup drive, depending obviously on the data you are backing up. While you cannot use the information directly from Finder, the GUI lets you see the file structure and you can easily restore a single file or the entire disk.

  • prefer not to make manual change to disk4 since disk4 is a simple "rsync -a" backup of disk5, so anything different will cause files to be sync again in the next run. – kenchew Aug 18 '15 at 9:22
0

Most (if not all) hard drives have bad areas on the disk. It is the nature of the medium that something that small cant be 100% perfect. So HD manufacturers build in code in the drive firmware to "map out" bad areas of the drive. Because of that two identical drives from the same manufacturer will have just slightly differing amounts of actual available space on them.

This is normal, accepted and quite common.

When purchasing drives it is wise to take this into account and select drives that are a bit bigger than you need to avoid running out of space, especially in conditions where the source drive is nearly full and the destination is the same rated size.

  • do you know any tool or command to check? 100G is a lot of bad areas. I had run Verify Disk on Disk Utility, both drive are reported fine. – kenchew Aug 19 '15 at 2:46
  • I know of no tool. 100GB is not that much when compared to the size of the drive (1000GB) ~1%. Large in absolute terms but otherwise small. You could replace it and hope for a better result on the next drive or return it for cash and buy a bigger drive. I would suggest the latter. – Steve Chambers Aug 19 '15 at 13:48
  • it's 10%. Anyway don't think it is a disk issue since the disk was usable just before it was formatted. It was actually NTFS formatted and used it the same manner, before reformatted to journaled,encrypted. In my update above, the disk again can contain all the files, so that should rules out physical disk issue. – kenchew Aug 20 '15 at 3:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .