4

I have my Macbook Pro (MacOS Mojave) connecting to an NFS share on a Linux machine directly connected via Thunderbolt with jumbo frames. Testing with iperf3, I can do ~14 Gbps (~1.8 GBps) over the connection, so I expect that network is not the bottleneck.

Here is the /etc/exports:

/srv/share         169.254.0.*(rw,sync,insecure,rw,all_squash,anonuid=1000,anongid=1000,no_subtree_check)

Doing some simple testing with dd, here is what I see:

Direct to disk on the server (~800MB/s)

~$ dd if=/dev/zero of=/tmp/testfile bs=5120 count=100000
100000+0 records in
100000+0 records out
512000000 bytes (512 MB, 488 MiB) copied, 0.62477 s, 820 MB/s

NFS from the server (~660 MB/s)

I connected a client on the server to see how it performed with a Linux client and taking the network out of the equation.

$ sudo mount -t nfs -o rw 169.254.0.1:/srv/share /tmp/share
$ nfsstat -m
/tmp/share from 169.254.0.1:/srv/share
 Flags: rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=169.254.0.1,local_lock=none,addr=169.254.0.1
$ dd if=/dev/zero of=/tmp/share/test_file bs=5120 count=100000
100000+0 records in
100000+0 records out
512000000 bytes (512 MB, 488 MiB) copied, 0.76516 s, 669 MB/s

From Macbook Pro over Thunderbolt (~150MBps)

I tried to match the default NFS client connection from Linux, but the Mac client doesn't support the same options. I've tried vers=3 and even async, but am never able to get it much faster than this.

$ sudo mount -t nfs -o rw,vers=4,rsize=1048576,wsize=1048576,hard,proto=tcp,timeo=600,retrans=2,sec=sys 169.254.0.1:/srv/share /srv/share
$ nfsstat -m
/srv/share from 169.254.0.1:/srv/share
  -- Original mount options:
     General mount flags: 0x0
     NFS parameters: vers=4,proto=tcp,hard,rsize=1048576,wsize=1048576,timeo=600,retrans=2,sec=sys
     File system locations:
       /srv/share @ 169.254.0.1 (169.254.0.1)
  -- Current mount parameters:
     General mount flags: 0x4000000 multilabel
     NFS parameters: vers=4.0,proto=tcp,port=2049,hard,nointr,noresvport,callback,negnamecache,nonamedattr,noacl,noaclonly,locks,noquota,rsize=65536,wsize=65536,readahead=16,dsize=32768,nordirplus,nodumbtimr,timeo=600,maxgroups=16,acregmin=5,acregmax=60,acdirmin=5,acdirmax=60,nomutejukebox,noephemeral,nonfc,sec=sys
     File system locations:
       /srv/share @ 169.254.0.1 (169.254.0.1)
     Status flags: 0x0
$ dd if=/dev/zero of=/srv/share/testfile bs=5120 count=10000
10000+0 records in
10000+0 records out
51200000 bytes transferred in 0.337533 secs (151688873 bytes/sec)

Given how fast the network is, it seems like I should be able to get 3-4x the speeds I'm seeing from the Mac. How can I improve the performance?

  • Are all your devices configured for the use of 'jumbo frames' because a mixture of devices configured for jumbo frames and devices not configured for jumbo frames on a network has the potential to cause network performance issues. – Dakta Moriamé Aug 31 at 0:38
  • Yes, both are using jumbo frames gist.github.com/ipmb/c8ca58662685c1b0846351b3102908d6 – Pete Sep 3 at 18:46
  • Have you tried another data transfer speed test? I find DD on macOS somewhat inaccurate in certain scenarios. – Thomas Jones Sep 4 at 0:43
  • I tried to run bonnie++, but the share consistently drops part way through the test. If I just cp a 512MB file, the timings are approx: Direct: 20ms, Local NFS: 180ms, NFS from Mac: 300ms. I also timed the rsync of a large .git directory and got: Direct: 0.2s, Local NFS: 1.2s, NFS from Mac: 3.5s. – Pete Sep 4 at 14:33
  • Pete, you need to use @ to notify people. one person per comment. – ankii Sep 4 at 20:29

You must log in to answer this question.

Browse other questions tagged .