I remember the times when I got my macbook pro with 4GB of RAM: I could launch Firefox, VirtualBox and Firefox inside a VirtualBox and it worked reasonably well.
Two years later, when I start the system and have no programs launched, 3.99 out of 4GB of RAM is used; and launching any tiny program, making ⌘ Cmd+Tab or a new tab in a browser is a huge pain for Mac OS X: I have to wait for dozens of seconds for any little action to complete.
However, if I sum up the "Memory" column in Activity Monitor, it would barely come up to 700 MB; Swap is low, File Cache does not exceed 300 MB.
So I don't really understand what could have happened with my Mac and what can I do to take it back to the times when I could use Firefox and Skype together without any slowing down.
Screenshot with all processes as suggessted in comments: