Why are all the homebrew formulas located in the 'libexec' folder?

Question

When I install packages manually, the typical layout looks like the following:

├── conf
├── lib
├── bin
│   ├── mvn
│   ├── mvn.cmd

But when I install it with homebrew, almost all the files are placed in the libexec folder:

├── bin
│   └── mvn
└── libexec
    ├── bin
    ├── boot
    ├── conf
    └── lib

Why is this strange layout used? It's a bit confusing when you read tool's official documentation and your paths mismatch.

Answer 1

The layout was a deliberate design decision on Homebrew’s side to keep the balance between usability, maintainer sanity, and not breaking things.

A comment in Homebrew’s source code says:

Installing JARs to "lib" can cause conflicts between packages. For Java software, it is typically better for the formula to install to "libexec" and then symlink or wrap binaries into "bin".

Why conflicts?

To explain what the specific conflict is and what causes it, let’s have a look at (a gross simplification of) what Homebrew does when you run brew install foo:

1. Homebrew downloads and unpacks everything foo needs according to the formula.

2. Homebrew follows the installation instruction according to the formula. For example, it copies files around, applies patches, and so on. In the end, you get a keg, the folder containing the fresh installation.

3. For your convenience, Homebrew looks at the bin and lib directories of your keg, then symlinks everything it finds into /usr/local/bin and /usr/local/lib so you don’t have to.

Note how in #3, the formula does not have a say. This helps keep complexity out of the formula definitions. A formula maintainer can never forget to create a symlink because she or he doesn’t have to; Homebrew does it automatically for everything that is in bin or lib.

The layout policy was (AFAIK) first codified in issue #678 of Homebrew’s legacy GitHub repository, which says:

This is expected behavior; libexec is assumed to be private-use stuff for that formula.

Basically, Homebrew considers the contents of bin and lib public artifacts, i. e. things that are supposed to be called from outside the formula.

Everything else, including the whole content of libexec, is considered an implementation detail, i. e. things for private use by the formula itself.

But what is the big deal with that? Could you, as an author of a formula like maven, not simply ignore those principles and instead live with a few unnecessary symlinks in /usr/local/lib, and give the user the same layout as the upstream project has?

Turns out that yes, you could, but it’s a horrible idea, and Homebrew maintainers would rightfully refuse such a formula until fixed.

Publishing private stuff to /usr/local/lib is a typical cause of JAR hell. (Imagine a Java-based Homebrew formula foo which uses mylib.jar v1, and another Homebrew formula bar which uses mylib.jar v2, with the version number not being part of the filename mylib.jar. Now go ahead and try to figure out why your business-critical application – which happens to use /usr/local/lib/mylib.jar because it’s in the path – only works on that 50% of your machines where foo has been installed after bar but not vice versa. This is an example of the conflict mentioned at the beginning.)

tl;dr Whatever you put into lib, Homebrew will symlink to /usr/local/lib for you. Putting a JAR you depend on into /usr/local/lib is a bad thing to do and leads to JAR hell.