2

As mentioned here the argument size of a command is quite limited. That's when you get zsh: argument list too long.

On my machine it seems to be

$ sysctl kern.argmax                                                                                                                                                                                                                            
kern.argmax: 1048576

So, already at 50k files with 25 character names, we have 1250000 and that's too much. Which seems a bit silly on a machine with 64 GB of RAM.

sysctl -a -W doesn't list argmax suggesting that it cannot be changed at runtime.

Can this limit be increased somehow instead of working around the issue with xargs/zargs/zcp/…?

12
  • 1
    man sysctl lists kern.argmax as “not changeable.”
    – Martin R
    Commented May 27 at 10:53
  • 1
    What is causing your argument list to grow? Do you need to exec or can you use built in? The xargs idiom is very well regarded (tested / known / performant) and specifically designed for handling data larger than $ARG_MAX such as common very long path-length with hundreds or thousands of files being sent through the same pipeline.
    – bmike
    Commented May 27 at 11:52
  • 1
    Sarien: What is the command you are trying to execute? It would seem to me you could use the find command with the -exec option to run your command on each argument. See the answer posted by nohillside. Commented May 27 at 14:58
  • I could, it just requires that I use more brain cells than I would have to with simple glob matching.
    – Sarien
    Commented May 27 at 20:49
  • 1
    Instead of increasing the limit and stumbling over a new limit later, how about fixing your program, so that it is not dependent on that limit anymore? Commented May 28 at 11:40

1 Answer 1

2

I feel your approach here is wrong. Use find and xargs as core programs designed to avoid overloading the call chain of the shell. Builtins are designed to handle long text without buffer overflow. For reasons exec can be abused for buffer overflow and has a fixed limit by design.

This is how dozens of engineers have designed shells and Unix for decades. You don’t have to like it, agree with it, but you are constrained by these implementation and security design choices in how the shell works.

2
  • The design choice was to have a limit and I am only asking to increase that limit. Note, the limit is not a compile time constant. I don't think I understand what the issue with exec and buffer overflows is that you're mentioning.
    – Sarien
    Commented Jun 10 at 7:39
  • You should be able to dig deeper here as both the shell and xnu are open source @Sarien github.com/apple-oss-distributions/xnu
    – bmike
    Commented Jun 10 at 10:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .