I consider this is a weird quirk of Microsoft Outlook. See here for more details: https://answers.microsoft.com/en-us/msoffice/forum/msoffice_word-mso_mac/disable-automatically-switch-keyboard-to-match/717be469-2267-483b-9d66-7f9e77fac01e
I guess I'll use the Outlook Webapp from now on sigh
edit: I just noticed it happens with other programs too. So maybe mac OS is the villain here. Hmm.