Skip to main content
13 events
when toggle format what by license comment
Aug 16, 2020 at 9:55 history edited Graham Miln CC BY-SA 4.0
Mention ad-hoc code signing to address issue in comments of @jvarela helpful answer.
Aug 13, 2020 at 10:34 history edited Graham Miln CC BY-SA 4.0
Missing word and clarification.
Aug 13, 2020 at 8:15 history edited Graham Miln CC BY-SA 4.0
Mentioned performance mode and HPC.
Aug 13, 2020 at 3:36 comment added Stuart Robbins The -m cProfile tool is nice. I've spent the better part of the last 9 hours using it to optimize this code chunk, in a test region getting the time down from 14.1 seconds to 3.8 seconds. The HUGE time sink was that my code uses two different conda environments (they don't play well together), and using "conda run -n [other] [code]" was causing a ≈0.5–1-second lag to just start it up (x3500 times). So, I have used other tools to work around those required scripts. However, while this full test area now runs in 15 minutes instead of 25, I'm STILL getting a 20–25% System usage cut, same PIDs.
Aug 12, 2020 at 17:33 comment added Stuart Robbins Looks like you edited again, so to address your question at the top: My goal is both. Obviously, running the code faster is the practical goal, but if there are coding issues running afoul of the way macOS operates that causes this, then I should know about those so as I proceed, I don't re-introduce the issue. Running it virtually in a different OS is not ideal, at all, and seems to be a cheat factor for an underlying issue.
Aug 12, 2020 at 12:09 history edited Graham Miln CC BY-SA 4.0
Fix grammar.
Aug 12, 2020 at 9:04 history edited Graham Miln CC BY-SA 4.0
Added performance profile link.
Aug 12, 2020 at 8:31 comment added Stuart Robbins You've made many additions to your post, so I'll just quickly note: I'm running everything on NVMe, so disk I/O is very fast. In the last 10 minutes, I've changed all the mv/rm/cp calls with os.system to built-in Python calls, but that did not change the overhead. The other programs that it calls are not something I have any ability to modify. This chunk of code has already been updated to minimize disk I/O, I can't change it further (I can change other aspects, but those aren't bogged down in this overhead). I'll let you know if Docker works, BUT in principle, it doesn't answer my question.
Aug 12, 2020 at 8:23 history edited Graham Miln CC BY-SA 4.0
More detail.
Aug 12, 2020 at 8:16 history edited Graham Miln CC BY-SA 4.0
More detail.
Aug 12, 2020 at 8:04 history edited Graham Miln CC BY-SA 4.0
added 396 characters in body
Aug 12, 2020 at 8:04 comment added Stuart Robbins I'll try this, I will see if it actually works right without the performance hit. The Linux test was with a slightly older version of my code.
Aug 12, 2020 at 8:02 history answered Graham Miln CC BY-SA 4.0