There's nothing wrong with turning off flash, but the main advantage is lower processor usage, not just memory consumption (though it's great to see turning it off has lowered Safari's footprint).
I don't think you need to worry about what Safari is doing with memory for a couple of reasons:
Safari 5 is a 64bit application, where as Firefox and Google Chrome are only 32bit (at least the mac public releases). This is relevant because 64bit applications appear to use more memory while they're running (The Growl team explains this better than I could, here).
If you've got free memory, which the OS (and Safari) can release quickly, safely and easily (speaking as a Programmer), why not use it to cache your web content - that's what it's there for. Trust your OS - it's clever, if another application needs the memory it'll just release the not-needed-right-now-bits-and-bobs.
Safari also has an unfair advantage- it's made by the same company that built your computer. I'm not saying Safari is infallible, or even that it's the best browser, just that you shouldn't over-think the statistics; memory is there to be used and if it's not negatively affecting your computing experience then don't stress.