1

I am running grep -o $string ~/giant_file where

string="foo
bar
baz"

This process keeps being killed after running for awhile and generating results, and I don't understand why. When it is killed, the following is output to the console:

[1] 69923 killed grep --color=auto --exclude-dir={.bzr,.cvs,.git,.hg,.svn} $string *

Is there a way to check the kernel logs to see why this is? In linux I would got to /var/log/kern.log to see if it is an OOM issue but I am not sure what to do on osx. If it isn't an oom issue, I am not sure what could be causing this, so other hypotheses are welcome.

  • What do you mean by "being killed"? Do you get a message of some kind, or does it just stop too early (before finding all occurances)? – nohillside Apr 2 at 18:50
  • @nohillside this to console [1] 69923 killed grep --color=auto --exclude-dir={.bzr,.cvs,.git,.hg,.svn} $string *. I am beginning to suspect there is some character in the file that is breaking it, have tried splitting it into pieces, some work some don't. – Andrew Apr 2 at 19:25
  • Just how big is giant_file? I believe macOS automatically kills processes when memory consumption passes some (very high) threshold. I once was trying to convert a 1 GB html file containing the entirety of the Encyclopedia Britannica to another format; all tools kept quitting partway through with killed: 9 after they'd used 40 GB of memory or so. – Wowfunhappy Apr 3 at 4:07
  • If you get the output you‘ve mentioned you seem to run Greg in the background. Can you please edit some details about how you call grep into the question? – nohillside Apr 3 at 4:25
2
  • Logs are in /var/log/System.log, you can also access them by running Console.app
  • Memory consumption of a process can be checked in Activity Monitory.app, at least as long as the process is running
0

The error message you see means that something else sent the grep command a kill signal. This does not usually happen by itself.

Sent signals are generally not logged to any log file, so you won't find it in /var/log/System.log for example.

grep doesn't use a lot of memory, so it is extremely unlikely to be due to a OOM issue. In addition, if it were, you wouldn't be getting the "killed" error message - instead grep would fail with one of these error messages: "malloc", "calloc" or "realloc".

The reason why your grep command is being killed seems to be that you have run it "inside" something else that manages it. It could be a Laravel worker queue, a Python pool system, a PHP worker system, etc. - there are many possibilities. You'll have to look at the documentation for the system you're using to get information on why it kills programs. Usually this is due to default timeouts.

  • hmm, I just ran it on the command line - not sure how it would be running inside of something? – Andrew Apr 3 at 1:21
  • Hmm that sounds odd then. Please include information about exactly what you did - otherwise we just have to guess what is wrong. Did you just start Terminal.app and then run that command? Sounds like you ran something before the grep command to get the $string variable set - so please detail the exact, full command lines you’ve run. – jksoegaard Apr 3 at 7:11
0

The first thing I'd try would be examining the return value.

$?  #in zsh, should be the same in bash. 

Since you are running this in the shell, you should be able to see it in launchd. Try:

launchctl list #the easiest invocation

You get 2 rows, first row is the PID, second is the exit status.

The proper invocation is:

launchctl print gui/$UID/  # for processes run in the gui. 

Same deal, running jobs have a pid, and the second column is the exit code. If it's negative then that's the signal that it caught.

You can also just hit CMD-I, and watch the thing run in the terminal inspector. Click on the gear to show the program arguments. I often watch Homebrew update this way.

Once upon a time ( pre SIP ) you could just use DTrace:

man kill.d # man -k DTrace to see them all

This would be the first thing I'd try, it'll tell you which process sent the signal. Alas, Apple has crippled this excellent collection of tools. It'll work for sure if you disable SIP, but if you have the stomach for it, you can drink directly from the hose:

sudo fs_usage -w  # So many things running. :(

You'll get your answer this way.

You'll see your grep die, and you'll see who killed it.

Good luck.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .