0

I have a launchd job that is supposed to run a php script at 6am every day. The script runs a number of backup-related tasks. After upgrading to Monterey, the task no longer starts at its scheduled time.

I use Lingon X to control it. When I run it manually from Lingon X as a "test", it starts, but sometimes does not finish. When I run the command from the terminal, the job starts and always runs to completion.

Here is the plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Disabled</key>
    <false/>
    <key>EnvironmentVariables</key>
    <dict>
        <key>PATH</key>
        <string>/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/jas/Keybase.app/Contents/SharedSupport/bin:/usr/local/sbin</string>
    </dict>
    <key>Label</key>
    <string>RH-backup</string>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/local/bin/php</string>
        <string>/Users/jas/Websites/RepHunter/cron/RH-backup.php</string>
    </array>
    <key>RunAtLoad</key>
    <false/>
    <key>StandardErrorPath</key>
    <string>/Volumes/Backup1/RH-backup/dbtemp/stderr.log</string>
    <key>StandardOutPath</key>
    <string>/Volumes/Backup1/RH-backup/dbtemp/stdout.log</string>
    <key>StartCalendarInterval</key>
    <array>
        <dict>
            <key>Hour</key>
            <integer>6</integer>
            <key>Minute</key>
            <integer>0</integer>
        </dict>
    </array>
    <key>StartOnMount</key>
    <false/>
</dict>
</plist>

There is never any output in either stderr.log or stdout.log.

The only suspicious pattern is that it appears that when started manually, the job often quits after a certain interval. Since the executable is a php script, the first thing I checked was whether the script was timing out.

As a test, I added set_time_limit(900) to the php script. That should not matter, as per the php documentation the script timeout does not include the time when system calls are made. The purpose of the script is to make numerous systems calls. So it should not be consuming a lot of run time on its own.

Plus this script has be working for several years without any such issues.

Any ideas?

EDIT 1:

After further testing I discovered some problems, but not the solution.

  1. Having the StandardOutPath and StandardErrorPath prevents the launchd job from running. We were thinking that it was because those paths were set to external drives was causing a problem. We just removed them completely. The script then runs.

  2. The script runs and calls all the external commands. There are a couple of calls to rsync to copy some files around including from remote servers. That works fine.

  3. Then the script calls duplicity to extract a file from the backup set. Duplicity immediately exits without extracting the file. That makes the rest of the functions in the script pointless.

  4. When the script is run from the command line, it always works as expected. It also worked as expected as a launchd job for over two years under Mojave and Catalina. Only after the upgrade to Monterey did the script lose its ability to write files.

  5. As a test, I gave Full Disk Access to duplicity. That makes no difference.

  6. Workaround is to run the script manually from the command line. That always works.

Why does Monterey block launchd jobs from writing files?

EDIT 2:

  1. As another test, I moved all of the intermediate files into /tmp/ with the hope that if a permissions issue might be revealed. That did not help.

  2. I added capturing of error output for the duplicity command. The following is the error output when run as a launchd job (sorry about the formatting--the code markup is not working for this text so I put it in as a quote):

Traceback (innermost last): File "/opt/local/bin/duplicity", line 117, in with_tempdir(main) File "/opt/local/bin/duplicity", line 103, in with_tempdir fn() File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/dup_main.py", line 1522, in main action = commandline.ProcessCommandLine(sys.argv[1:]) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/commandline.py", line 1158, in ProcessCommandLine config.gpg_profile = gpg.GPGProfile() File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpg.py", line 92, in init self.gpg_version = self.get_gpg_version(config.gpg_binary) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpg.py", line 108, in get_gpg_version res = gnupg.run([u"--version"], create_fhs=[u"stdout"]) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpginterface.py", line 376, in run process = self._attach_fork_exec(gnupg_commands, args, File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpginterface.py", line 429, in _attach_fork_exec self._as_child(process, gnupg_commands, args) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/duplicity/gpginterface.py", line 468, in _as_child os.execvp(command[0], command) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 574, in execvp _execvpe(file, args) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 616, in _execvpe raise last_exc File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 607, in _execvpe exec_func(fullname, *argrest) FileNotFoundError: [Errno 2] No such file or directory

GPGError: failed to determine gnupg version of None from b''

EDIT 3:

Searching on that error message "GPGError: failed to determine gnupg version ...", I found a page at https://bugs.launchpad.net/duplicity/+bug/1865427. That page suggested that for cron jobs adding --gpg-binary with the path to gpg.

I tried that, but the launchd job still did not work.

FINAL EDIT

Prior to the Monterey upgrade, gpg was on the default path used by the launchd job, for whatever reason. Configuration changes due to the OS upgrade caused it no longer to be on that default path. So previously it was not necessary to adjust the path for the launchd script, but now it is. Fixing that up solved this issue. So marking this question as answered.

2
  • From point 1 in the edit: I'd recommend pointing StandardOutPath and StandardErrorPath to someplace on the system volume (I generally use something in /tmp), and see if they show any informative messages. Jul 29 at 2:19
  • I was planning to do this if I should ever need those files again. Aug 2 at 23:55

2 Answers 2

0

Total guess: Could the problem stem from macOS's change to Zsh instead of Bash as its default shell? That change came in Catalina, and I seem to recall issues with my launchd scripts at the time... Could you have kept Bash as default back then, and now Monterey's launchd expects Zsh?

OK, now I got curious. Were you aware of this article in the Apple documentation?

"Updating helper executables from earlier versions of macOS", at https://developer.apple.com/documentation/servicemanagement/updating_helper_executables_from_earlier_versions_of_macos

3
  • Thanks for the input. The problem with launchd itself was worked around by eliminating the stdout and stderr. Now the problem appears to be that duplicity cannot access its files when run under launchd, whereas it has no problems when run from the command line. So it does not seem to be a shell issue. Plus I had no problems in Catalina; only in Monterey. The article about helper executables seems to not apply. Aug 2 at 15:40
  • @JeffreySimon did you found a solution? Nov 19 at 5:13
  • Solution found. See final edit to question. yesterday
0

SOLUTION

Prior to the Monterey upgrade, gpg was on the default path used by the launchd job, for whatever reason. Configuration changes due to the OS upgrade caused it no longer to be on that default path. So previously it was not necessary to adjust the path for the launchd script, but now it is.

That solved the problem.

3
  • gpg is not an Apple supplied program so any changes are in the users configuration not OS changes . I would not as you are using /opt/local I assume you are on Macports - if you read its migration notes it says delete the current version and instalkl the new version for the OS.
    – mmmmmm
    yesterday
  • Thanks for your comment. The point was the previous launchd did not require a path setting; the new one did. So it was not something we were prepared to look for. What this might be implying is that it is good to reinstall all your MacPorts components after an OS upgrade. However I have never understood that to be a requirement in the past. yesterday
  • According to macports documentation it is a requirement trac.macports.org/wiki/Migration
    – mmmmmm
    18 hours ago

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .