2

I have been using multiple units of WD 5TB external hard drive with my macOS 10.13.6 on a 2018 MacBook Pro 15-inch.

I realised that it is almost always the case that read/write speeds drop when the hard drive is going to be full (approx 100+ GB remaining). For example, it usually takes 20 minutes to fully copy a 32 GB microSD card over, but it then takes the same time to copy only 16 GB when the hard drive is nearing full. I must say that sometimes it can go incredibly low, like now it's taking 39 minutes to transfer only 14 GB, and to the best of my knowledge there's no other file read/write from the hard drive.

Sometimes rebooting the machine will help to restore the original read/write speeds, but not always.

  1. Is such a drop in read/write speeds expected and normal?
  2. I partition my hard drive into 10 partitions on GUID Partition Map, 500 GB each, HFS+, Case Sensitive, Journaled. Will having fewer partitions with larger sizes per partition help me avoid the abovementioned problem? (I can't use APFS because of backward compatibility requirements.)
  3. How might I better understand what's going on "under the hood" to help me troubleshoot this problem eg. how might I use command-line tools or GUI tools to help me to confirm whether the source of this issue is really with the hard drive, and that there are no coincidences.
  4. Any additional suggestions on what might be the root of this issue?
7
  • Could it simply be caused by fragmentation? I would guess that there won‘t be a continuous sequence of blocks free once the drive is nearly full, so any bulk writes will be slow.
    – nohillside
    Nov 25, 2023 at 12:34
  • @nohillside is it possible that this (ie fragmentation) almost always happens with one model of harddrive ie amazon.sg/dp/…, but not another like amazon.com/Western-Digital-Desktop-External-Drive/dp/B01LQQHKZY ? This is my experience. Only the former harddrive model slows down.. Nov 25, 2023 at 12:39
  • Different drive models can have different behavior, yes.
    – nohillside
    Nov 25, 2023 at 12:52
  • @nohillside if it's really fragmentation, is there a way to get macOS to write files to the harddrive more strategically to prevent this recurrent problem from occurring? Also, can I take any measures manually to prevent that as well? eg. does the number of partitions play a role? and if I write some files to partition 1 then jump to partition 5 then partition 3 then jump back, will it cause more fragmentation than filling up "from start to finish" ie fill up partition 1 then fill up partition 2 then 3 etc.? Nov 25, 2023 at 13:42
  • It's probably a combination of drive speed, drive size, drive firmware, partition size, filesystem format, access patterns (especially writes) etc. Having several partitions for sure doesn't help, because each fills up separately. Why do you create partitions at all, HFS+ supports 5 TB? Also, if you know that this specific disk model gets slow when it is filling up, why not pick another one?
    – nohillside
    Nov 25, 2023 at 14:02

1 Answer 1

1

I would absolutely consider buying more space for speed. Not filling a disk is guaranteed to keep speed high. The seek time to get from piece of data to piece of data can slow down write speeds and different sectors of the disk have faster or slower absolute write speed.

I would also recommend you make fewer partitions on HFS+ so that the defragmenting and hot file optimizations can free up the fastest band of the disk for files to be written and files accessed the most frequently.

Your case of dumping files might allow the system to fill the slowest corners of the disk with "stale data" and let the active data be on the fastest part of the disk - reducing your wait. Once you fill the drive, you will guarantee you experience the slowest possible response as the last "slow corners" get filled with data.

The best OS for measuring, creating logical volumes where you can steer the storage that I know of is AIX. Here is some excellent reading from their disk tuning guide. The physical characteristics of the disk bands are universal - it's build in to the drive cache, head seek mechanism and drivers. Each OS has differing plans to optimize things - so many of the AIX commands are not possible for HFS since it's not quite as developed as IBM's implementation of a storage stack.

Hard Disk band speeds

You can use the iostat command on macOS similar to the description IBM provides (and also be sure you use vm_stat 5 in another window to be sure you are I/O bound and not another CPU / paging issue is causing contention) in your baseline and test measurements.

iostat -n 5 disk0 

And change n and keep adding your physical disks by number (use diskutil list to enumerate them once they are attached) so you can watch the IO on every disk attached and see if you are blocked and then try an experiment.

Overall, no amount of "advanced knowledge" is going to make a huge difference - spending money on larger disks, letting the system take advantage of the entire disk or buying faster disks if you need to fill a disk past 80% is the end goal. If you're on the tipping point, you might be able to free up some space or measure a different choice of fewer partitions and see if the system can manage performance (macOS does try to manage things and not require us to tune IO like AIX allows and needs tuning for large storage systems).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .