To answer this question accurately, you will need to get some statistics on how much RAM you are currently using. But in general, more RAM is going to improve things, especially if you're still on 4GB.
On the other hand, the 2009 iMac is really reaching EoL at this point -- according to Mactracker, the support status for that machine is currently "Vintage". The biggest factor for this machine's performance in my opinion is the processor architecture -- Core 2 Duo had a good run, but the Core ix architectures are really in another class.
To put it another way -- one of my machines is a 2010 iMac 3.6 Ghz Core i5 with 16 GB of RAM, and it's still completely usable at this point. Despite it being only one year off from your model, the processor architecture difference is what's allowing my machine to stay usable while yours is showing its age. (I'm sure the 16GB RAM is helping too, but I think the processor architecture comparison is more relevant.)
Might be time for an upgrade! 6-year lifespan is nothing to sneeze at; I'd call that getting your money's worth.