6

I am using moviepy to automate clips to make a video. I am using python 3.10. When I run from pycharm, it is using performance cores to render my videos. But when I moved to crontab to schedule this task, it is running on efficiency cores which is taking 3 - 4 times the normal time.

How can I force the python script to run on performance cores?

Processor Usage

1 Answer 1

5

I believe this is the inverse of this question. Cron itself is running on efficiency cores because it's a background task, and your Python script inherits its priorities. taskpolicy is a front-end to the setpriority and getpriority APIs, which, luckily, are exposed to Python directly, so you should be able to import os; os.setpriority(...). The specifics of the parameters you need to pass are probably specific to your particular application though; it looks like you've got a bunch of processes and you want to run at a high priority, so read the relevant documentation and see if adding to your script will enable you to re-elevate your own task priority from the background.

Note this language from man setpriority on macOS, all emphasis mine:

Additionally, the current thread or process can be placed in a background state by specifying PRIO_DARWIN_THREAD or PRIO_DARWIN_PROCESS for which. Only a value of zero (the current thread or process) is supported for who when setting or getting background state. prio is either 0 (to remove current thread from background status) or PRIO_DARWIN_BG (to set current thread into background state). When a thread or process is in a background state the scheduling priority is set to the lowest value, disk IO is throttled (with behavior similar to using setiopolicy_np(3) to set a throttleable policy), and network IO is throttled for any sockets opened after going into background state. Any previously opened sockets are not affected. The getpriority() call returns 0 when current thread or process is not in background state or 1 when the current thread is in background state. Any thread or process can set itself into background state.

Note that the DARWIN values, i.e. PRIO_DARWIN_PROCESS, are platform-specific constants that are not exposed by Python, so I wrote a brief C program to extract them:

#include <stdio.h>
#include <sys/resource.h>

int main(int argc, char** argv) {
    printf("PRIO_DARWIN_THREAD %d PRIO_DARWIN_PROCESS %d PRIO_DARWIN_BG %d\n", 
           PRIO_DARWIN_THREAD, PRIO_DARWIN_PROCESS, PRIO_DARWIN_BG);
    return 0;
}

and then I wrote this Python script to illustrate how to use them, which alternates between background and foreground modes:

from os import setpriority, getpid

PRIO_DARWIN_THREAD  = 0b0011
PRIO_DARWIN_PROCESS = 0b0100
PRIO_DARWIN_BG      = 0x1000

from time import time

from itertools import cycle

seconds = float

def burn_cpu(how_long: seconds) -> None:
    start = time()
    while time() < (start + how_long):
        pass

for (human, machine) in cycle([("background", PRIO_DARWIN_BG),
                               ("foreground", 0)]):
    print("in", human, getpid())
    setpriority(PRIO_DARWIN_PROCESS, 0, machine)
    burn_cpu(60.0)

I ran it as python3 backgroundr.py & python3 backgroundr.py & python3 backgroundr.py & python3 backgroundr.py in my terminal to ensure that it was using enough parallelism to properly show up on the CPU usage graph, then took this screenshot illustrating it very clearly moving between the performance cores and the efficiency cores:

graph showing alternating efficiency core and performance core usage

4
  • I have set the priority with the following code and it is still using E Cores. os.setpriority(os.PRIO_PROCESS, os.getpid(), 19) Commented Jul 14, 2022 at 15:22
  • You should be using 0 to indicate the current process, and 0 for the priority to remove from background status, as the manpage indicates.
    – Glyph
    Commented Jul 15, 2022 at 19:49
  • OK I substantially expanded the answer to include a fully working example. It's possible that cron, specifically, does something to forcibly stick its subprocesses to efficiency cores (I think "E cores" and "P cores" is Intel terminology which is why I've been avoiding it, but gosh apple should let us abbreviate:) ) but otherwise these APIs seem to work fine.
    – Glyph
    Commented Jul 15, 2022 at 20:20
  • @krishnamraju Does this fix your problem?
    – Glyph
    Commented Sep 10, 2022 at 4:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .