What am I doing that causes zsh to silently change to vi mode?

Question

I'm a software developer, and as such I frequently use Terminal.app. I've been using it for years. At some point in the last week or two, I started encountering this problem somewhat frequently; I've never seen it prior.

I use zsh as my default shell. I'll be typing out a complex command. I hit return, the command executes, and I see that it wasn't exactly what I wanted. I hit the up arrow, then arrow left to the part I want to change, and I use delete/backspace to clear out the part I want to change, but nothing happens. I also cannot type any characters.

Today, I found out that when this happens, zsh has changed to vi mode. I can hit i to switch to insert mode, x to delete a character, etc. Further, I can restore something resembling normalcy by typing set -o emacs. Prior to this discovery, I had just been abandoning the Terminal window in question and starting anew. Frustrating!

I am not sure what I might be doing to kick zsh into vi mode. I am definitely not typing set -o vi! It has to be either a bug or a very simple hotkey combination that I could be hitting by accident.

This only seems to happen to the local zsh on my Mac. If I'm ssh'd into a remote Linux machine, using bash or zsh on the remote machine (I've changed my default shell on some but not all of our servers), it doesn't seem to happen.

Furthermore, I've discovered that when I open a new Terminal window, it doesn't appear to be in vi or emacs mode. It behaves in a vaguely emacs-y way, but control+A doesn't jump to the beginning of the line, and the small handful of other emacs-specific keystrokes I'm familiar with also don't work. If I run set -o emacs, then those things start working.

If I run set -o | egrep "on$" (which lists the currently enabled zsh options) in a fresh terminal, neither emacs nor vi appears in the results.

I looked through my update/install history in the App Store, and I think this might have started when I installed Xcode. (I mostly develop Java stuff, and didn't have it installed previously, although I did have the developer tools installed.) It does not appear to correlate with the installation of any system-level software updates.

I did some googling and found no evidence of anyone else encountering this problem.

If it matters, I'm using:

• Late 2013 27" iMac.
• 32GB, 1TB SSD, 3.5-3.9GHz i7.
• MacOS High Sierra 10.13.4
• zsh 5.3 (x86_64-apple-darwin17.0)
• Xcode 9.3.1 (installed, but I'm not using it when any of this happened)

So, my questions:

1. does anybody know what I might be doing to cause the silent switch to vi mode?
2. is there a setting I can change so that this stops happening?
3. is there some logging or other diagnostic output I can enable to make it more obvious when this happens?
4. what is this non-emacs, non-vi input mode I'm seeing in a fresh Terminal window / zsh process called?

Answer 1

Sigh. Exploring my own situation further, I think I was hitting escape. I think zsh was in vi mode all along, but more specifically, it was in vi's insert mode! If I go to a brand new editor and hit escape, I find myself in vi's command mode.

If that's the case, I'm still interested to know why set -o | egrep "on$" doesn't indicate that we're in vi mode.

EDIT: double-sigh. I originally wrote my question with all references to bash, having completely forgotten that years ago I changed my default shell to zsh, which is mostly the same. I went back and edited the question to be more clear, and then googled again, replacing bash with zsh. I quickly found this: https://unix.stackexchange.com/questions/197839/why-does-exporting-vim-as-editor-in-zsh-disable-keyboard-shortcuts

Lo and behold, my .profile has EDITOR=vi, which I put there about two weeks ago. I removed that line, and hitting escape no longer puts me in vi command mode.

Better yet, I followed the instructions given on that answer and was able to restore my EDITOR setting without borking up zsh. Yay! :)