What is the fastest (most performant) method to generate MD5 checksums on macOS?
I'm currently using the stock /sbin/md5 and it takes an order of magnitude longer than generic Linux on a similar spec'd host.
1 Answer
I'm currently using the stock /sbin/md5 and it takes an order of magnitude longer than generic linux on a similar spec'd host.
This is likely because the MD5 command line utility is included as part of the OpenSSL distribution. Of which, the latest version of macOS is using v0.9.8. You can find this on opensource.apple.com under the macOS 13.2 Release (scroll down to find OpenSSL098-84.120.1 and you can click on the GitHub link to take you directly to the source).
By comparison, FreeBSD 13.2-RELEASE (my other OS of choice) is utilizing OpenSSL version 1.1.1t. I would bet the Linux distro you're using likewise, utilizes a more recent version of OpenSSL. 3.0 and 3.1 were only released in November 2022 and March 2023 respectively (no v2, BTW), so it's not likely to be widely distributed yet.
You can upgrade OpenSSL by installing it via MacPorts or Homebrew (or go to version 3) Be aware that there are caveats because it doesn't replace your existing OpenSSL, but installs alongside it.
md5digest at a time? Is the input file or memory based? Are you launching a new instance ofmd5for every digest? Adding this information to your question will help attract better answers.