Go to `~/Library/Application Support/CloudDocs/session/containers`, and delete everything you don't want to keep (make a backup first).

And all the hidden folders at the iCloud Drive Folder should be gone.