2

I have a standard syslog server running on a LAN host and listening on UDP port 514. I'm trying to have my Mac (running macOS 12.4 with SIP enabled) ship logs in syslog format to this remote server.

I was able to figure out that if I edit /etc/syslog.conf and add e.g. the following line:

# send all logs to 192.168.10.10
*.*  @192.168.10.10:514

and then bounce syslogd:

sudo launchctl kickstart -kp system/com.apple.syslogd

That I do see some logs being sent to the server. However, many log entries are missed because of Apple's new Unified Logging System (ULS). I am aware of /etc/asl.conf which also has some relevant logging config, but I was unable to figure out if or how this could be used to ship logs to a remote machine.

I also am aware of the log command, and can use it to query and output in syslog format, e.g.

log show --info --predicate 'messageType > 2' --last 60s --style syslog

I thought about using this to periodically (via a LaunchAgent) query and ship the logs using netcat or some other simple tool. But this seems quite inefficient, as I'd need to parse each line and store a hash somewhere to see if that message had already been sent to avoid duplicates. Also, some log entries span multiple lines, so parsing with things like sed or awk become cumbersome.

Without adding too many 3rd party tools to the mix, does anyone have a recipe for sending a more complete set of logs to a remote server?

1
  • This is a bit of an X Y question - what are you doing with the logs once you have them? You might have far better options putting the intelligence on the end point and not trying to orchestrate it all on your syslog server. Or - maybe a follow on question identifying one specific message you want to go to a file and then we can pick apart how to change that part of logging to have it both in the database and on a file you can push externally…
    – bmike
    Jun 28, 2022 at 16:25

2 Answers 2

2

I would like to point out that there are quite readily available, easy to use and free 3rd party tools for solving this problem:

In particular, the remote_syslog2 utility is relatively easy to setup as it will collect logs from the Unified Logging System (as well as other sources) and send them off to a remote syslog server.

If you want to not use third party tools, you can start on the command line by getting a stream of logs from the ULS in syslog format like this:

log stream --style syslog

And then script that using the built-in Perl and its Sys::Syslog module.

4
  • Where did I say "no 3rd party tools" ? I did say I wanted to minimize the use of them, but I understand that this is not a simple problem to solve and may require it.
    – luckman212
    Jun 29, 2022 at 15:45
  • Ah - sorry about that! I misread it as no third party tools, but I see you just wanted to minimize the number. In any case, the answer is in my answer - just use remote_syslog2.
    – jksoegaard
    Jun 29, 2022 at 15:52
  • Well it wasn't easy. I spent the whole day on this and ended up with a working solution. I'm using log stream to pipe ULS logs in ndjson format to a flat file, which is then feeding remote_syslog2 and that in turn is sending syslog messages over TCP to a Synology running syslog-ng (LogCenter) with a custom parser. I created a LaunchAgent to starts this all up without any user intervention. Thanks for the guidance. If anyone wants more detail, upvote this and I'll see about doing a full writeup.
    – luckman212
    Jun 29, 2022 at 23:10
  • Easy is always relative ;-) I bet the next time you go to set this up on a Mac, it will be easy for you.
    – jksoegaard
    Jun 29, 2022 at 23:24
3

The core problem you have is Apple unified the system logging and the files no longer contain the entries you need. It’s a highly structured database with TTL / verbosity / relevance baked in to the logging database structures.

Diagram courtesy of EHN & DIJ Oakley © 2022 - https://eclecticlight.co/2020/02/07

This means you may want to rethink your consumption of the files or how you filter the data that’s important to you.

Short term, you’ll want to change the logging configuration to dump the events you need to log files of your choice so you can maximize the value of your old central logging platform.

You will need extensive expertise on a team to avoid third party tools for recent macOS versions, there’s no easy fix here and you are on the correct path. Hopefully thinking about what you really need and when you need it will help guide your efforts.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .