22

How can I update the whatis database?

$ sudo /usr/libexec/makewhatis
Password:
makewhatis: /usr/share/man/whatis.tmp: Read-only file system

I believe being able to update this database will solve some other issue I'm having. My path to discovery as follows...

I recently started noticing that fish shell completions were annoyingly slow on my machine, possibly shortly after upgrading to Catalina.

I did a little profiling with fish -d5 and noticed that the majority of the time was spent on the apropos command. I did some reading and learned that the tools apropos, whatis and makewhatis are all related. They index man pages and make them searchable. Fish shell is (correctly) using them to offer helpful completions.

When I run whatis or apropos standalone, I get the following output:

$ whatis man
hugo-gen-man(1)          - Generate man pages for the Hugo CLI
groff_man(7)             - groff `man' macros to support generation of man pages
groffer(1)               - display groff files and man~pages on X and tty
man(1)                   - format and display the on-line manual pages
man.conf(5)              - configuration data for man
zshall(1)                - the Z shell meta-man page
xml2man(1)               - MPGL to mdoc (man page) translator
makewhatis: /usr/lib/./libgutenprint.2.dylib: No such file or directory
makewhatis: /usr/lib/libsasl2.2.0.1.dylib: Not a directory
makewhatis: /usr/lib/libldap.dylib: Not a directory
makewhatis: /usr/lib/libsqlite3.0.dylib: Not a directory
makewhatis: /usr/lib/libcom_err.dylib: Not a directory
...

Followed by at least 100 more lines of the "Not a directory" messages. I believe it is all these useless lines that are slowing things down.

So I thought maybe I just need to rebuild the whatis database (perhaps after the Catalina upgrade?). However, it doesn't seem to work:

$ sudo /usr/libexec/makewhatis
Password:
makewhatis: /usr/share/man/whatis.tmp: Read-only file system

So this part is a little disturbing. How can I rebuild the whatis database? I have a hunch this will solve my issues if I can figure it out.

4
  • Please don't add answers directly to the question, post it as an answer below instead so that people looking for answers find it where they expect to find answers.
    – nohillside
    Nov 8, 2019 at 6:37
  • @nohillside I don’t have an answer. There’s a hack to work around a side effect. But no answer to the actual question. Nov 9, 2019 at 12:16
  • Well, it is a solution, at least a temporary one. It's worthwhile to share.
    – nohillside
    Nov 9, 2019 at 13:58
  • After struggling with this issue multiple times (from Big Sur to Monterey), I believe that turning /usr/share/man/whatis into a symlink which points to a file in the user home-directory is the most reliable solution to this problem: apple.stackexchange.com/a/426551/79496
    – GDP2
    May 19 at 22:17

8 Answers 8

12

The following can be used as a workaround for the macOS 10.15.1 version of the apropos command, wherein it spews out complaints of the form makewhatis: /usr/lib/lib … .dylib: Not a directory.

First create the workaround script:

$ mkdir -p ~/workarounds
$ sed -e 66s@/usr/lib@@ /usr/bin/apropos > ~/workarounds/apropos.macos_10.15.1 
$ diff /usr/bin/apropos ~/workarounds/apropos.macos_10.15.1 
66c66
<     for d in /var/cache/man $manpath /usr/lib
---
>     for d in /var/cache/man $manpath 
$ chmod +x ~/workarounds/apropos.macos_10.15.1

Next add an alias to your shell to tell it to use the workaround script, until a newer version of the canonical script becomes available.

For Zsh you can use the following command:

$ /bin/cat <<END >> ~/.zshrc
# Workaround for broken apropos command.
alias apropos=~/workarounds/apropos.macos_10.15.1
END

For other shells such as ksh or bash, use ~/.profile or ~/.bash_profile, as appropriate.

What does the workaround do?

Apropos requests (and man -k requests) are handled by the /usr/bin/apropos script. That script searches for “whatis” database files in all the directories of the man path (see man —path), plus /var/cache/man and /usr/lib. The checks for /var/cache/man/whatis and /usr/lib/whatis appear to be there for historical reasons, however those files are not actively generated in Mojave or Catalina. A lot of different people have contributed to the various flavours of Unix over the years, and many of them had different good ideas about where to put different types of files. At some point in time, somebody decided that /usr/lib would be a good place to put a whatis file, and at some other point somebody else figured that /var/cache/man would be a good place. Others thought that the appropriate place would be the respective man page directories. Different solutions that seemed appropriate at the time. The apropos script has traditionally checked those locations in case a whatis file was present.

With the move to making the system directories on Catalina read-only (a good move), whatis database files can not be written to directories such as /usr/share/man. There are different ways that Apple could handle that, but for this release somebody decided to alter the apropos script by making it generate results on the fly by calling /usr/libexec/makewhatis.local for any man page directory that does not contain a whatis file.

That new apropos code works fine for actual man page directories, and for /var/cache/man (since it doesn’t exist), but it fails for /usr/lib. The workaround detailed above just eliminates /usr/lib from the list of directories searched.

As a final step, set yourself a reminder for a month or two from now to check whether Apple has fixed the apropos script. If so remove your workarounds, by removing the alias and the workaround script.

8
  • 2
    If only apple would hire contractors or place bounties so people could fix things like this for us all. +many - thanks for explaining precisely what Apple broke on the unix side in making their read only filesystem.
    – bmike
    Nov 20, 2019 at 12:12
  • 1
    I just ran in to this problem (makewhatis: ... Not a directory) with Catalina, and was really happy to see this well-explained fix here. Thanks. Nov 20, 2019 at 16:32
  • I have in the past tried making critical partitions within other flavours of Unix read-only. It is a good idea from a security standpoint, but hard to do. There are many instances where partitions that could theoretically be made read-only, have elements that expect to be writeable. Eventually I quit following that policy because it required too many workarounds. So I am happy to see Apple’s move to locking down the system as an O/S policy, I just hope that they don’t keep it locked down so tight that they prevent self-patching of the system.
    – Hambly
    Nov 20, 2019 at 21:01
  • 1
    It's disappointing that this didn't get fixed in the macOS 15.2 update. Dec 16, 2019 at 20:32
  • 1
    This seems fixed in macOS 10.15.4. I couldn't find any announcement that confirms this. However, maybe this was fixed by changing makewhatis. The date /usr/libexec/makewhatis was last modified is now "Mar 17 11:42".
    – minopret
    Apr 13, 2020 at 20:50
3

I just ran into this and googled my way here...

Looks like "whatis" will either grep through generated whatis files, or generate them on the fly to the stdout. What we're seeing is the output of "makewhatis" being run on /usr/lib.

You'll get the same errors from:

/usr/libexec/makewhatis -o /dev/fd/1 /usr/lib

/usr/lib isn't in the manpath (output of "man --path") - it's added explicitly by "whatis", though for what reason I have no idea. There are no man pages there, and makewhatis is clearly expecting everything in a man folder to be a subdirectory.

If we could edit the "whatis" script, we could fix it. But we can't, because /usr/bin is read-only.

If we could generate an empty /usr/lib/whatis, the complaints would stop. But we can't because /usr/lib is read-only.

It might be possible to fix /usr/libexec/makewhatis.local to stop this nonsense, but it's read-only.

I need to do some research to see if there's a way to get the OS volume mounted read-write for a bit.

On a related note: Even if we did get a "makewhatis" to run successfully, it won't generate /usr/lib/whatis, because /usr/lib not in the man path... so it won't fix this. Creating an empty /usr/lib/whatis is probably the easiest and safest option, if we can figure out how.

2

Regarding a Solution that would generate the missing whatis files:

The solution to updating the "whatis" database in /usr/share/man requires a fix from Apple. They need to either add /usr/share/man to their list of firm links (similar to the implementation for /usr/share/snmp), or add a static copy of the whatis file to the system volume.

Firm links are a new feature of the APFS; designed to support merging read-write volumes with read-only system volumes. As of the Catalina release the core operating system files are held on a read-only volume, which is then merged with a read-write data volume via the use of firm links. Under macOS version 10.15.1, /usr/share/man is only present on the read-only system volume. You can add an entry for /usr/share/man to the data volume by creating the directory /System/Volumes/Data/usr/share/man, as demonstrated by klanomath’s answer, but it will not be mapped onto the system directory (/usr/share/man) until a corresponding firm link is created.

A list of the current firmlinks can be found in /usr/share/firmlinks. The documentation that I have been able to dig up thus far is not clear as to whether firmlinks is a reference file, or a configuration file, but it looks to me like a configuration file that is read and utilized as part of the boot procedures. Assuming that it is a configuration file, then you could theoretically correct the issue by adding an entry for /usr/share/man to the file.

Unfortunately, since /usr/share/firmlinks is housed in the read-only system volume, you can not edit it as a user, or even as the super-user. Even in single-user mode, mounting the system volume group in read-write mode is prevented (ie: /sbin/mount -uw /does not work). It may be possible to r/w mount the the system volume as a subsidiary drive on a secondary system, and then make the edits; but that is more experimentation time than I was willing to put in.

So in short, the improved security of Catalina prevents updating that directory until Apple fixes the issue.

The above notes are relative to Catalina (macOS v 10.15.1). As it is a simple fix, I expect the issue will be corrected soon.

2
  • Unfortunately adding another line in the firmlinks file is not sufficient to get a firmlink (tested this much earlier already). Another step on the file system level is necessary to link both folders to get a working firmlink.
    – klanomath
    Feb 23, 2020 at 0:08
  • This appears to be the best explanation for what I am seeing with LaTeX. The macOS distribution makes a symlink in /Library/TeX/texbin to the actual 'active' install in ../.DefaultTeX/Contents/Programs/texbin which allows the user to switch releases. It also adds a file in /etc/paths.d that points to the softlink. All man invocations complain that the symlink isn't a directory.
    – flip
    Nov 13, 2020 at 18:49
2

I just found this issue today and created the following aliases in ~/.zshrc to clean up the command's output:

alias apropos="apropos 2>/dev/null"
alias whatis="whatis 2>/dev/null"

The aliases remove the errors from the output by using redirection. The shell has two file descriptors for output. The standard output is file descriptor 1 and standard error output is file descriptor 2. The errors being generated by the makewhatis.local script are being sent to the standard error output.

Redirecting the standard error output is done by using the stderr file descriptor “2”, the output redirection operator “>”, and the destination file “/dev/null”. The /dev/null file is a special filesystem object that discards everything written into it. With the errors redirected only the desired results are displayed.

3
  • It’s helpful if you explain what those commands do.
    – Allan
    Feb 22, 2020 at 3:40
  • That's a workaround, but not a solution.
    – Mojo66
    Jun 10, 2021 at 4:26
  • I agree it is a workaround and not a fix. It seems the problem has been corrected, at least for the original "whatis man" command. Running that command on macOS version 11.4 did not produce any errors when I ran it.
    – Tim
    Jun 11, 2021 at 17:08
1

Try this one (😎):

  1. sudo mkdir /System/Volumes/Data/usr/share/man
  2. sudo /usr/libexec/makewhatis -o /System/Volumes/Data/usr/share/man/whatis
  3. user@host ~ % cd /System/Volumes/Data/usr/share/man/
    user@host man % lsl
    total 384
    drwxr-xr-x  3 root  wheel  -          96 Nov  3 01:33 .
    drwxr-xr-x  4 root  wheel  sunlnk    128 Nov  3 01:32 ..
    -rw-r--r--  1 root  wheel  -      160236 Nov  3 01:33 whatis
    

    lsl is an alias for ls -laOe@ on my system

Fun facts:

  • I don't know where this file is (except that the file is there) – the file can't be found with sudo find / -name "whatis" in the file system
  • The file survives a reboot
  • I have no clue whether this file is used at all by whatis/apropos/fish|bash|zsh shell completion (and solves your problems)
1
  • Nice idea, but unfortunately whatis/apropos don't seem to use the watis file placed in /System/Volumes/Data/usr/share/man Nov 4, 2019 at 2:27
0

Catalina broke the man command.

To ignore the error messages under /bin/bash, create an alias for man:

alias man='/usr/bin/man 2>/dev/null'
0

I believe this is fixed in MacOs 10.15.4. Thanks to @minopret for pointing this out in the comments on the original question. Running the unmodified whatis or apropos commands do not result in errors.

0

My own workaround to this problem consisted of creating a symlink at /usr/share/man/whatis. Since I use Emacs, I often rely on the M-x man command, which in turn uses apropos for auto-completion entries. With a broken whatis file, I can't get proper auto-completion results, which is very inconvenient.

I've also hitherto used a root cronjob script to keep my whatis file up-to-date, as follows:

*/5 * * * * /usr/libexec/makewhatis -o /usr/share/man/whatis /usr/share/man /Applications/Xcode.app/Contents/Developer/usr/share/man [etc. ...]

However, this no longer worked since /usr/share/man/whatis is no longer directly writable. For me, it was also not even present in the filesystem path any longer.

So, to solve this problem, after much investigation and consideration, I decided to create a symlink at /usr/share/man/whatis instead of an actual file, pointing to a writable file under my home folder: /Users/user/.emacs.d/whatis

I created this symlink by using this utility: https://github.com/fxgst/writeable_root

Specifically, I ran:

$ ./writeable_root
$ cd usr/share/man
$ ln -s /Users/user/.emacs.d/whatis whatis

And then the system rebooted, with my new symlink in place.

I could not use synthetic.conf to create this symlink, as synthetic.conf only creates symlinks directly under /. It will not create them in any lower directories in the filesystem hierarchy.

That way, I can have this file editable and I simply changed the cronjob to be run by my own user instead of as root, with the simple change that it modifies the file in my home folder instead of at /usr/share/man/whatis:

*/5 * * * * /usr/libexec/makewhatis -o ~/.emacs.d/whatis /usr/share/man /Applications/Xcode.app/Contents/Developer/usr/share/man [etc. ...]

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .