You most probably do not want to avoid macOS from suspending apps when the system runs out of RAM when you know that the only alternative is to kill the program. I.e. stop the program from running now and here without saving its data or anything like that.
However, you do not have to restart the programs that are paused as in "close the program and start it over". Instead you can actually unpause the program when you have sufficient free memory on the system. You can do this from the terminal by running:
kill -CONT <pid>
where should be replaced with the process ID of the process, you want to unpause.
You can find process ID using Activity Monitor or the "ps" command in the Terminal (in this case, look for processes with status T as these are the paused/stopped ones).
An easy way of unpausing and finding the ID in one go is:
kill -CONT `pgrep myprogram`
where myprogram is the name of the program you want to unpause.
The second part of your question where you describe the ideal scenario is very hard to accomplish with a standard program. In general programs can request memory from the operating system, but the operating system cannot request it back afterwards. It is possible, but it requires very specialized programs which is generally not the case.
This means that once macOS have granted your program for example 1 GB of RAM, the operating system cannot just take it back when other programs want more memory. If you take away memory from programs that are not specially crafted to allow that, they will effectively crash as soon as they attempt to use the lost memory.
