6

I have tried increasing the max open files limit in mac by running the following command

sudo launchctl limit maxfiles 524288 10485760

When I try running ulimit -n it shows 524288

But while running the same command in IntelliJ idea's terminal it shows 10240

  1. I've tried reinstalling IntelliJ idea
  2. I've tried restarting IntelliJ after invalidating cache and restarting
  3. I've restarted the system multiple times.

Please let me know how to fix this issue.

2
  • Considering the default value is 256 what do you need those very high figures for
    – mmmmmm
    May 15, 2021 at 17:27
  • 2
    Trying to build a huge Java based project on IntelliJ. It keeps failing because of the too many open file error
    – Shylajhaa
    May 15, 2021 at 17:29

2 Answers 2

9

The problem is caused by IntelliJ being a Java program that is subject to the constraints of the Java virtual machine, which by default on macOS is set to limit the number of open files to 10240 - no matter which setting is used elsewhere in the operating system.

The solution is as follows:

  1. Add -XX:-MaxFDLimit in IntelliJ Idea (Help->Edit Custom VM options)
  2. Invalidate cache and restart IntelliJ

This removes the arbitrary 10240 limit on the Java virtual machine, and makes it so that IntelliJ is instead only limited by the general macOS open files limit - like any other program.

4
  • This is awesome! Excellent outcome - I’m glad it didn’t require extra changes to the system limits and it was just your IDE not requesting the system limits.
    – bmike
    May 16, 2021 at 14:11
  • IDEs do not and cannot request system limits, that’s not how it works. The system limits are always applied no matter what - that’s why they’re system limits. In this case Java was applying its own, stricter limits. The fix was for Java not to apply its own limits.
    – jksoegaard
    May 16, 2021 at 17:01
  • I just did this, and still get a 256 ulimit inside PyCharm. I've cleared caches, rebooted the machine. Other Terminals have 1000000, PyCharm just 256. Is PyCharm different? Mar 3, 2023 at 22:03
  • It appears that the JVM will use the soft limit (default 256) if this flag is added. questdb.io/blog/max-open-file-limit-macos-jvm/#conclusion
    – euclio
    Nov 12, 2023 at 14:28
0

Most processes would start closing files well before it reaches 10,000 open files - perhaps the best fix is to rein in the tool. Loops can run away faster than you adjust an open file limit in my experience.

None of the things you tried will help or would be expected to help if the IDE sets its own limits rather than relying only on system limits.

If you want to do heavy background processing on your Mac, you should consider enabling macOS Performance Mode: https://support.apple.com/en-us/HT202528

Enabling macOS Performance Mode changes a number of system settings (also settable with sysctl) to lift limits and otherwise optimise the system for heavy-lifting instead of normal desktop usage.

You might also consider that it might be faster to virtualise another unix or Windows than tuning macOS to your needs.

4
  • 1
    This answer does not make sense - you're basically saying that instead of building a large project, she should stop building large projects, use a different operating system - or go on some sort of self-education marathon. The right answer is provided by Shylajhaa herself, namely to ensure that the ulimit is set for the Java build subprocess started by IntelliJ instead of for the Terminal in general.
    – jksoegaard
    May 16, 2021 at 11:46
  • Or it’s a starting point since an hour after the question OP had someone asking if the IDE just needed to grab the system limits. It’s Ok if you don’t see how “wrong answers” can be useful or even lead to correct ones when they have an opinion on how to get started. No worries if my answers aren’t your cup of tea @jksoegaard and I appreciate you stating your objections clearly and I also appreciate the -1 whomever voted.
    – bmike
    May 16, 2021 at 14:09
  • I just think this isn’t an answer - it’s a comment. I like almost all of your answers, but this one I didn’t like - I hope that’s okay.
    – jksoegaard
    May 16, 2021 at 16:59
  • Yes x very OK, seriously appreciate the criticism on this. Comments are for clarifying the question and I’m not trying that hence the “answer” to address the topic.
    – bmike
    May 16, 2021 at 17:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .