Changing the Key Modifiers (CMD<->CTRL) affects VM

Question

I changed my ⌘CMD to behave like ⌃CTRL and ⌃CTRL to behave like a ⌘CMD, but this affects also my VirtualBox VM.

So I do ⌃CTRL+C for Copy in Mac, but I gotta do ⌘CMD+C to copy in Ubuntu...

That's pretty confusing, is there a way to go around this?

Answer 1

I saw that VMWare has this possibility, VirtualBox not.

I know use a headless VM (No GUI) so I use the OSX Terminal for all operations, behavior stays the same.