I write programs to do calculations. In order to automate some things I setup my Mac to consume a queue of jobs and a cron will check the queue and then run them through.
I noticed that cron jobs run so much slower.
As root I am running all the commands at nice -n -100 (yes I know -20 is the limit)
if I run this command from the terminal it take 62 seconds. Checking the ps this is what I see:
UID PID PPID F CPU PRI NI SZ RSS WCHAN S ADDR TTY TIME CMD STIME
0 17826 17195 4006 0 31 -20 413217088 581472 - R<+ 0 ttys000 16:11.07 /usr/local/bin/M 0.358 12:41PM
Now comparing that to the cron'd job this same calculations will take 441 seconds. The ps comes out to:
UID PID PPID F CPU PRI NI SZ RSS WCHAN S ADDR TTY TIME CMD STIME
0 18231 18230 4004 0 20 -20 411514512 515632 - R< 0 ?? 28:16.72 /usr/local/bin/M 12:47PM
This is 7x slower!
Both processes are at -20 NICE, but the priority for the cron job is 20 while running from terminal is 31. So I am guessing that priority is the mitigating factor here.
I can't seem to figure out how to change the priority of the job. Lots resources out there just seem to think by changing nice you also change priority. But that is not the case.
The question I have is how do I change the priority of a cron job (or any job in fact).
I am running on a MacBook Pro (2013). I was thinking maybe Apple Mac Pro desktops could be setup differently but I just don't know.
First off, on the performance gap you're seeing, this test (probably) ran entirely on a P-core (performance):
if I run this command from the terminal it take 62 seconds
While this test ran on an E-core (efficiency):
the cron'd job this same calculations will take 441 seconds
That decision was driven by the general context of each job- when Terminal.app was the parent, it was classified as "interactive" ("should fun fast"), while "cron" running the same job is "background/utility" ("take your time").
That leads to here:
As root I am running all the commands at nice -n -100 (yes I know -20 is the limit)
The standard Unix commands for process priority have never* really "fit" how macOS actually works and the evolution of the system has only increased these issues. Practically speaking, macOS simply isn't a command line oriented operating system** and, because of that, has never made "external" process priority management a priority. Our focus has always been about how apps prioritize their work and how the system manages the overall workload, not external prioritization (like "nice").
*Literally, "never". I've seen people been complaining about "nice" not doing the "right" thing since MacOS X 10.0 shipped in 2001.
**This focus has driven many low level choices throughout the entire kernel/system. For example, when compared to many other operating system, the system's scheduling quanta is much smaller and the scheduler greatly prioritizes minimizing I/O latency over CPU bound work. Both of those choices reduce the latency of user input and broader I/O, at the expense of overall processing throughput. When people comment that macOS feels "snappier", this is one of major drivers of that.
All this matters because it's important to understand that this:
This is 7x slower!
...was an intentional choice, NOT an accident. There are changes you can make that would will alter that behavior, but one of the systems implicit "goals" is to make work the user can't see "slower", both to reduce power consumption and to ensure maximize the resources available for work the user CAN see.
The question I have is how do I change the priority of a cron job (or any job in fact).
A few different answers:
-
The standard/expected approach here is that processes will manage their own threads QOS using the relevant API, as described in "Prioritize Work..." sections of the "Energy Efficiency Guide for Mac Apps".
-
I haven't looked very closely at the details, but posix_spawn does have the ability to set QOS at process creation, however, it only allows the lower priority QOS levels to be set.
-
In terms of "external" controls (like "nice"), the only other option I'm aware of is "taskpolicy". I don't think it will work any better than "nice" for direct process priority, but changing I/O policy could have an impact.
__
Kevin Elliott
DTS Engineer, CoreOS/Hardware
