3

Note: I am looking for an answer for Catalina and for Big Sur. The commands below were run on Catalina. Most related questions and answers are for older versions of macOS which work a bit differently, but even then I have not found an exact answer to my question.

As I explained here, I want to increase the ulimit -n (max open files) for the Terminal program when launched from the Finder or when automatically started when logging in as part of restoring open windows. Even though I have performance mode turned on, the default ulimit -n is only 256, and most Terminal windows take up 2 file slots: 1 for the shell and one for the "Saved Application State" file, limiting me to around 128 windows, which sometimes is not enough.

The only way I have found to change the ulimit for the Terminal is via

sudo launchctl limit maxfiles <soft-limit> <hard-limit>

I would like to set the soft limit to something bigger than 256 but still reasonably small, like 512 or 1000. I would like to leave the hard limit unlimited. Unfortunately, I found out that you cannot set the hard limit to "unlimited", but that is OK, I can set it to an appropriately high number. I checked the kernel limits (remember, I have performance mode enabled):

$ sysctl kern.maxfilesperproc kern.maxfiles
kern.maxfilesperproc: 600000
kern.maxfiles: 1200000

It seemed reasonable to me to set the hard limit to kern.maxfilesperproc, so I ran

sudo launchctl limit maxfiles 999 600000

at which point several programs immediately crashed. I found that this did not just set the ulimit, it also set the kernel parameters, and not the way I expected:

$ sysctl kern.maxfilesperproc kern.maxfiles
kern.maxfilesperproc: 999
kern.maxfiles: 600000

So programs with more than 1000 files open crashed. Obviously, those programs knew they were going to need to have a lot of files open and adjusted their resource limits appropriately, which is fine, and I need to keep that. On the other hand, programs like Terminal can get into loops that consume resources until they run out, so I want to keep soft limits for those programs low.

I tried setting ulimit in /etc/bashrc and /etc/zshrc but those only affect their respective shells, not commands like Terminal that are launched by the Finder/launchd.

Is there some way I can adjust the default soft limit for number of open files (ulimit -Sn, setrlimit()) for Terminal (or system-wide) without also changing the kernel parameter kern.maxfilesperproc (sysctl)?

1 Answer 1

3
+150

This answer requires a bit of explanation about how these limits actually work:

In general these types of resource limits come with a soft limit and a hard limit. The soft limit means that a process will receive a signal notifying it whenever it exceeds that limit. The hard limit means that the process cannot exceed this limit no matter how hard it tries.

A process is allowed to change its own resource limits at runtime. The soft limit can be raised up to the hard limit, and lowered as much as wanted. However, the hard limit can generally only be lowered. Only the superuser ("root") is allowed to raise the hard limit.

The limit on the maximum number of simultaneously open files is enforced by the kernel. Whenever a process tries to open a file (or duplicate a file descriptor, etc) the kernel will check whether this means that the process exceeds the current limit. If so, the process will either receive a signal (for the soft limit) - or the attempt to open/duplicate/etc will fail (for the hard limit).

Note that this means that when you change the limit during runtime, nothing really happens besides the kernel storing the new limit in its data structures. If the limit is lowered and that means that processes are now exceeding their new limit, nothing happens. The processes will not receive a signal for the soft limit, and files will not be closed or the process terminated for the hard limit. However, when the process attempts to open new files, they will either receive a signal or the open attempt will fail.

Another important property of these resource limits is that they're inherited when a new process is forked. This means that when a process with a limit of X starts a new child-process C1, that child-process C1 will also have a limit of X. If the process later changes its limit to Y, and starts a new child-process C2, this child-process C2 will also have a limit of Y.

The macOS kernel is structured so that the lowest level is a Mach kernel - on top of this is layered a BSD subsystem for enabling a Unix userland. Note that these resource limits only apply to processes in this BSD subsystem of the kernel. All the processes you normally interact with on a daily basis are such BSD processes.

During boot of the system, the Mach kernel initializes the BSD subsystem in a function known as bsd_init(). In this function, the kernel sets the default (i.e. starting) limits for new processes. Among these it sets the default soft limit for number of open files per process to be 256. This value is hard coded into the kernel executable and cannot be configured on runtime. This is why on a default system, you'll see that launchctl limit will report maxfiles as 256.

You can configure that value by compiling your own kernel. Again, this will typically wreak havoc with various integrity protections such as Secure Boot, but it is doable. After downloading the kernel source code, but before compiling it, you'll want to change the file bsd/sys/param.h in approx. line 101, which says:

#define NOFILE  256  /* default max open files per process */

Note that this will change the default limits for not only Terminal.app, but also everything else.

In addition to the limit values stored by the kernel in its data structures for each process, there's also a set of matching kernel parameters. In this scenario, that's for example the kern.maxfilesperproc parameter that can be changed with sysctl. This parameter sets the limit for the first process created in the system - i.e. the root of the inheritance tree. Even though the process is running with superuser privileges, it won't be able to change its own resource limits using setrlimit() above that ceiling (without first raising the ceiling using sysctl).

In Unix-like systems and thus also BSD systems, this specific process is generally known as the "init" process. Historically this was a program named initd. On macOS the role of the init process is actually performed by launchd.

This is why when you use launchctl to set a new limit for maxfiles, you actually end up changing these kernel parameters. Afterall, those are the limits to be enforced for the init process (launchd). As they're stored as kernel parameters, instead of in a per-process data structure, they will be persisted even if the init process (launchd) should crash and be restarted. launchd is itself responsible for this, as it doesn't just call the usual setrlimit() system call to change its resource limits when you use the limit parameter with launchctl - it actually also runs sysctl() directly.

launchd being the root of the inheritance tree means that when you change its limits, this will actually have an effect on all processed to be started on the system. As you want to target the Terminal process specifically, this is not what you want.

If you take a look at Activity Monitor, search for the Terminal process and click the info-button - you'll see that it actually lists launchd as the direct parent of Terminal. This means that there's nothing "in between" launchd and Terminal whose resource limits you could have changed to get an effect on Terminal.

On Linux it is possibly via the prlimit() system call (commonly using the command line tool prlimit) to change the resource limits of an another given process. Sadly, Linux is quite alone in having this system call - it is not found on other Unix systems, including macOS. If we had this system call on macOS, you could just use a tool like prlimit to set the limits you wanted for the ´Terminal`.

You could possibly try changing the resource limits for the program that ultimately runs Terminal.app for you. For example you could change /System/Library/LaunchDaemons/com.apple.WindowServer.plist and add a SoftResourceLimits key with a dictionary holding a NumberOfFileskey set to the integer value of the limit you want. However that limit would apply to every process run by the graphical subsystem, and not only Terminal.app.

All this boils down to the fact that if you want to change the resource limits for Terminal specifically, you need to get the Terminal process to do that itself (i.e. it needs to call setrlimit() itself). Unfortunately there's no option in its Settings or anything like that to set this resource limit.

That leaves you with only one clear solution: You need to change the code of Terminal in order to set the resource limit. That is often done using so called "code injection". This could for example take the form of injecting a dynamic library using DYLD environment variables. However, it is generally a pain to do this on more modern macOS systems, as you will be hindered by SIP (System Integrity Protection) - and various other types of protections, especially on Apple's own programs. It is however doable if you're willing to disable these protections.

Commonly it is not the worth the time, effort and risk involved - and it is easier to simply set the general resource limit so high that you can open the number of Terminal tabs you want - even though that also means that other processes will have a similar higher limit.

A practical work-around might be to simply copy Terminal.app so that you can have 2 sets of Terminals running. It's not perfect, but it would allow you double the amount of tabs without changing any system settings.

You could also live with having to launch Terminal.app via another program that sets the resource limit. For example you could create a small shell script that sets the limit and then launches Terminal.app. It would mean that this shell is kept around while you have the Terminal open, but might be worth it for your scenario.

6
  • This is a lot of interesting information, but it skirts over the part I am most interested in. It cannot be an absolute requirement that launchctl gets its soft limit from the kernel, because at startup, kern.maxfilesperproc: 600000 but lauchctl sets a soft limit of 256. Where does the 256 come from and (how) can I change that without changing the kern.maxfilesperproc?
    – Old Pro
    Jul 11, 2021 at 0:03
  • I didn't say it was an absolute requirement that launchctl gets its limit from the kernel - I wrote that when you set it, you also set the kernel parameter. So when you "launchctl limit maxfiles aaa bbb" you will always get the kernel parameter changed also (unless you're running a seperate copy of launchd that is not the init process, but that is not the case on a standard macOS system). The same argument applies for launchd itself as for Terminal - as there's no official way to set the resource limit for another process, you have to get launchd to set it itself. As launchd always [...]
    – jksoegaard
    Jul 11, 2021 at 9:16
  • [...] sets the kernel parameters when setting its own resource limit, you can't do that without injecting code into launchd - which breaks the system integrity protection. You could change the resource limits for the program that ultimately runs Terminals for you - the parent is launchd after it is launched, but it will be kicked off by something run by WindowServer, so you could set the resource limits on WindowServer in the launchd configuration for WindowServer... however that is essentially changing the limits for almost everything you run on the system, so it also can't target Terminal.
    – jksoegaard
    Jul 11, 2021 at 9:18
  • @OldPro I have added an explanation for where the 256 figure comes from, and how it can be changed.
    – jksoegaard
    Jul 11, 2021 at 9:37
  • I should be able to use a shell script or an Automator script to do this, shouldn't I? It could change it's limit from 256 to 512 or 999 or whatever, then launch Terminal, and it would default to that new limit, right?
    – Old Pro
    Jul 14, 2021 at 21:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .