I have a launchd job that is supposed to run a php script at 6am every day. The script runs a number of backup-related tasks. After upgrading to Monterey, the task no longer starts at its scheduled time.
I use Lingon X to control it. When I run it manually from Lingon X as a "test", it starts, but sometimes does not finish. When I run the command from the terminal, the job starts and always runs to completion.
Here is the plist:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Disabled</key> <false/> <key>EnvironmentVariables</key> <dict> <key>PATH</key> <string>/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/jas/Keybase.app/Contents/SharedSupport/bin:/usr/local/sbin</string> </dict> <key>Label</key> <string>RH-backup</string> <key>ProgramArguments</key> <array> <string>/opt/local/bin/php</string> <string>/Users/jas/Websites/RepHunter/cron/RH-backup.php</string> </array> <key>RunAtLoad</key> <false/> <key>StandardErrorPath</key> <string>/Volumes/Backup1/RH-backup/dbtemp/stderr.log</string> <key>StandardOutPath</key> <string>/Volumes/Backup1/RH-backup/dbtemp/stdout.log</string> <key>StartCalendarInterval</key> <array> <dict> <key>Hour</key> <integer>6</integer> <key>Minute</key> <integer>0</integer> </dict> </array> <key>StartOnMount</key> <false/> </dict> </plist>
There is never any output in either stderr.log or stdout.log.
The only suspicious pattern is that it appears that when started manually, the job often quits after a certain interval. Since the executable is a php script, the first thing I checked was whether the script was timing out.
As a test, I added set_time_limit(900) to the php script. That should not matter, as per the php documentation the script timeout does not include the time when system calls are made. The purpose of the script is to make numerous systems calls. So it should not be consuming a lot of run time on its own.
Plus this script has be working for several years without any such issues.
After further testing I discovered some problems, but not the solution.
Having the StandardOutPath and StandardErrorPath prevents the launchd job from running. We were thinking that it was because those paths were set to external drives was causing a problem. We just removed them completely. The script then runs.
The script runs and calls all the external commands. There are a couple of calls to rsync to copy some files around including from remote servers. That works fine.
Then the script calls duplicity to extract a file from the backup set. Duplicity immediately exits without extracting the file. That makes the rest of the functions in the script pointless.
When the script is run from the command line, it always works as expected. It also worked as expected as a launchd job for over two years under Mojave and Catalina. Only after the upgrade to Monterey did the script lose its ability to write files.
As a test, I gave Full Disk Access to duplicity. That makes no difference.
Workaround is to run the script manually from the command line. That always works.
Why does Monterey block launchd jobs from writing files?
As another test, I moved all of the intermediate files into /tmp/ with the hope that if a permissions issue might be revealed. That did not help.
I added capturing of error output for the duplicity command. The following is the error output when run as a launchd job (sorry about the formatting--the code markup is not working for this text so I put it in as a quote):
Traceback (innermost last): File "/opt/local/bin/duplicity", line 117, in with_tempdir(main) File "/opt/local/bin/duplicity", line 103, in with_tempdir fn() File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/dup_main.py", line 1522, in main action = commandline.ProcessCommandLine(sys.argv[1:]) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/commandline.py", line 1158, in ProcessCommandLine config.gpg_profile = gpg.GPGProfile() File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpg.py", line 92, in init self.gpg_version = self.get_gpg_version(config.gpg_binary) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpg.py", line 108, in get_gpg_version res = gnupg.run([u"--version"], create_fhs=[u"stdout"]) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpginterface.py", line 376, in run process = self._attach_fork_exec(gnupg_commands, args, File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpginterface.py", line 429, in _attach_fork_exec self._as_child(process, gnupg_commands, args) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpginterface.py", line 468, in _as_child os.execvp(command, command) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 574, in execvp _execvpe(file, args) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 616, in _execvpe raise last_exc File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 607, in _execvpe exec_func(fullname, *argrest) FileNotFoundError: [Errno 2] No such file or directory
GPGError: failed to determine gnupg version of None from b''
Searching on that error message "GPGError: failed to determine gnupg version ...", I found a page at https://bugs.launchpad.net/duplicity/+bug/1865427. That page suggested that for cron jobs adding --gpg-binary with the path to gpg.
I tried that, but the launchd job still did not work.
Prior to the Monterey upgrade, gpg was on the default path used by the launchd job, for whatever reason. Configuration changes due to the OS upgrade caused it no longer to be on that default path. So previously it was not necessary to adjust the path for the launchd script, but now it is. Fixing that up solved this issue. So marking this question as answered.