How can I make Caps Lock work like it did prior to Sierra?

Question

Downloaded and installed macOS Sierra (10.12) on 10/1/16.

Since then, my Caps Lock key is behaving weirdly different (I'm an OS X user since Mavericks).

Note: This only seems to affect users with more than one language as their input source.

Explanation: Caps Lock simply does not work when a (update: Non-Latin language) language other than English (in my case, Hebrew, an RTL language) is set as the input language. When that's the case and Caps Lock is pressed, it simply blinks for a second and then turns off again, but then and turns off. Even when I am able to turn it on, it doesn't have the same effect (all keys behave as if Caps Lock is off)

Why is this bad: Just a small example — take a look at the bottom-right key, which is ? (or / when Caps Lock is on). When Hebrew is selected, it's actually a period (.) and that is normal. Prior to Sierra, I could press the Shift key to get a ? and hold Caps Lock to get /. Right now I can't use / when I'm typing in a language other than English because Caps Lock doesn't work.

Note that Caps Lock works just as expected when English is set as the input source.

Answer 1

I think I may have found the solution:

It appears that in Sierra, the "default" behavior that I described above is possible, and is now "a feature". Only problem is it's turned-off by default (which is plain ludicrous).

There is now a new option in the input sources screen, which, when turned on, reverts the behavior to the desired one.

The Caps Lock key longer turns green when non-Latin languages are used, but the keys behave as expected. There's the annoying symptom of accidental switching between languages for those who use Caps Lock to Capitalize in English. Luckily for me at least, I switched to Shift for that early on.

Answer 2

You can have the old behavior back if you create a duplicate Hebrew keyboard with Ukelele and use that instead of Apple's