You don’t actually specify what model of Mac you have, nor what you mean by within a session.
However, the perspective warp function is a rather CPU and GPU intensive task. How long it takes will depend on a number of factors, such as the complexity and size of the files you’re working with, the amount of VRAM your GPU has access to, and so on.
Also, if you’re trying to use the perspective warp function repeatedly during relatively short periods of time, you may find that your system starts heating up beyond acceptable limits and the CPU and/or GPU become throttled in order to protect them from damage.
Finally, while you do seem to have a good amount of RAM (32GB is nothing to be sneezed at), the perspective warp feature is actually more demanding of the GPU and the amount of VRAM it has access to. So you may find that’s the cause of your memory issues, especially if you’re using perspective warp repeatedly within short periods.