Since you won’t be running an abstract workload - making abstract generalizations won’t be super useful but my experience is you always want to get the most modern CPU and Mac to run your workload since the processors of the last 2-3 generations have the best compiler optimizations and the amount of work that can be done per CPU cycle increases far more than a back of the envelope calculation would show.
Let’s assume you already have both the iMac and the Mac Pro - your time will be better spent working to tune the workload for your iMac or have it run in the cloud rather than run on a bunch of old Mac that need management / orchestration / etc.
If you haven’t purchased hardware, I’d say neither your equiment is the best. If you have a workload you can run on GPU you would need something like bitcoin mining rigs or go to the cloud again and instead spend your money on a MacBook and displays and learning / consulting and not hardware.
To come back to your specifics:
- RAM is never the real bottleneck - RAM makes up for a less optimized solution, but you need storage and a smart algorithm to get out of the way of CPU and storage bottlenecks.
- if you have a massively parallel CPU load - get it on dozens to hundreds of CPU that you rent or are super low cost and run Linux or some OS you can orchestrate with chef or puppet or ansible or some of the newer cloud tools to let software define the OS configuration (google cloud / amazon cloud / https://bosh.io/docs/ / https://www.spinnaker.io/ )