4

I'm using rsync in Mac OS X Lion to copy a backup directory from my local HD to a network mounted share (local LAN). My goal is to archive the local backup to the network drive, thereby freeing up local disk space.

The rsync command I am using is:

rsync -aPW * /Volumes/netshare/path

For some reason, my free disk space is decreasing as rsync runs, and I eventually run out of space. After killing the rsync process and waiting, the consumed disk space is eventually freed and returns as free space.

It seems as if the copied files are being cached locally (even though they're already on the local drive) prior to being copied by rsync. I've checked and it's not swap space, and I have 1.5 GB of free RAM as rsync is running.

Has anyone else noticed this and is there a way to disable this caching in rsync?

Update:

I've tested with rsync, cp, and copying a file in Finder, and free disk space decreases in all 3 scenarios, so it's a general OS issue. I've been using lsof to see which files rsync has open, and nothing there appears to be growing in size. My theory has been that the files are being cached somewhere on the drive as the copy progresses. I have > 2GB RAM free and 0 swap space being used, so it doesn't appear to be a virtual memory usage issue.

As I mention below, disabling local snapshots also had no effect on the growing disk space.

Improve this question
2
  • That's definitively strange. I've copied 130 GB last week the same way you did and didn't run into any problems. Sorry for asking again: Are you absolutely sure that /Volumes/netshare is on the share? Try ejecting the share and have a look at /Volumes/netshare afterwards.
    – nohillside ♦
    Oct 12, 2011 at 18:18
  • I ejected the network share and there's no trace of it remaining in /Volumes. I'm using Time Machine on a different external drive, but I disabled local snapshots. In another forum it was suggested that one should disconnect external drives to prevent this fluctuating space issue when copying. So I tried that, but free disk space still decreased.
    – argoneus
    Oct 13, 2011 at 1:11

4 Answers 4

Reset to default
2

My first guess would be that you got a typo in the target path (/Volumes/netshare/path, especially the netshare part) and are actually duplicating the data onto the source drive.

If not, I don't really see the benefit of using rsyncfor a one-off copy operation with limited risk of interruption -> just use cp instead, OSX takes care of the network part anyway if you copy something to a share mounted under /Volumes.

If you absolutely want to use rsync leave off the -PW options for a starter. -W is used anyway unless you sync directly to another server by specifying the target as remotehost:path/to/dir, -P is only needed if the sync runs the risk of getting interrupted.

Improve this answer
6
  • 1
    Welcome to Ask Different. If the answer helps, I wouldn't mind getting an upvote and/or an accept :-)
    – nohillside ♦
    Sep 30, 2011 at 18:01
  • Thanks for the answer. /Volume/netshare/path is of course an example, not the actual path. The reason I'm using rsync is I'm copying ~182GB of data and want to see progress, which by default cp doesn't show. rsync was suggested here as a good alternative to cp, but one that shows progress.
    – argoneus
    Sep 30, 2011 at 18:07
  • 1
    Nevertheless, take the target path you are using and run an ls on it. rsync doesn't cache/buffer anything unless you specify --cache, so I still think something else is going wrong here.
    – nohillside ♦
    Sep 30, 2011 at 18:25
  • If I do an ls in the path, I see the actual contents of the network share. Also, I can see the drive activity light pulsing :-) So I know it's actually accessing the drive's content, not some phantom local version.
    – argoneus
    Oct 1, 2011 at 1:05
  • You could also try using --progress instead of -P. -P also includes --partial which is something you won't need. On the other hand: If you just want to see progress, you can easily use cp -v or copy the files directly in the Finder.
    – nohillside ♦
    Oct 1, 2011 at 18:40
0

Possibly this?

http://rdutoit.home.comcast.net/~rdutoit/pub/robsoft/pages/backup.html

  1. If you are backing up to an external drive, check the hidden Volumes folder. This is a known problem when copying files to an external disk. Sometimes, if you pull the plug accidentally on a drive, or for no apparent reason, the Volumes folder will contain left over aliases of drives that are no longer present. This can result in data being copied into them instead of your desired disk, thus your hard drive mysteriously fills up with "hidden" backups! To check the folder go to the "GO" menu in the finder menu bar and select "Go to folder" and type "/Volumes" in the search filed. This will bring up the hidden folder. I have included a menu item called "Show Volumes folder" in the Window menu, or the Help menu if you are in Background mode.

(backup list is a gui wrapper to rsync)

Improve this answer
1
  • Hmm, interesting. I see the network share in /Volumes, and from command-line I can ls the contents. I can also browse from /Volumes and go into the share, viewing the contents in Finder.
    – argoneus
    Oct 1, 2011 at 1:14
0

I think I figured it out via http://www.reedcorner.net/news.php/?p=238. Mac OS X Lion caches Time Machine backups to the local HD, in a new feature called local snapshots. You can disable this using

sudo tmutil disablelocal

No more wonky, frequently fluctuating free disk space!

Update:

False alarm, apparently. Disabling local snapshots didn't solve the problem, and the disk space is still being consumed as rsync progresses. Large files copying result in a greater amount of disk space consumed. I've gone from 4.35 GB free down to 1.5 GB, then I killed rsync.

Improve this answer
1
  • So it didn't have to do anything with rsync or cp at all?! Oh well...
    – nohillside ♦
    Oct 4, 2011 at 14:31
0

Finally figured it out. The network share was mounted via WebDAV. Thinking this could be the issue, I created an SMB mount (of the same remote share) to test. Sure enough, the files transferred without resulting in decreasing free disk space! So the problem lies somewhere in Mac OS X WebDAV client. Apparently files are being cached locally in the processes of being copied to the remote share.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .