After a fresh restart, everything is smooth. During the next couple of days of regular usage of the system, a problem of responsiveness gradually increases, until you can count a full second between you swiping a desktop and the system actually responding to the gesture.
I've tried reducing memory pressure by closing high memory apps, but it doesn't help at all.