Cannot ssh into machine?

Question

I have a Mac that for some reason I cannot manage to ssh into anymore. I'm completely stumped and am looking for ideas of what to try next (short of reinstalling the whole system). I've tried the following:

• Of course I've made sure that "Remote Login" is enabled in the sharing system prefs on the server.
• The Firewall is turned off in the server's Network settings.
• I've double-checked that the ssh service is active by typing sudo launchctl list | grep ssh which gives the output - 0 com.openssh.sshd just like on the machines that I am able to ssh into.
• In the terminal of the server, if I type ssh localhost it works as expected (i.e. I get a shell). Also, typing telnet localhost 22 gives the normal behaviour, so the port appears to be active.
• I've done a tcpdump on the server which shows incoming requests on port 22 but no response whatsoever back to the client. (Unless I listen on the loopback interface and log in from the server itself, in which case everything behaves as one would expect.) The output of the tcpdump on the problematic machine is as follows (the machine with an address ending in 14.85 is the problematic one):

% sudo tcpdump -i en0 -n host XXX.XXX.14.4
Password:
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on en0, link-type EN10MB (Ethernet), snapshot length 524288 bytes
15:24:43.556929 IP XXX.XXX.14.4.49744 > XXX.XXX.14.85.22: Flags [S], seq 1150830320, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 3381388029 ecr 0,sackOK,eol], length 0
15:24:44.558314 IP XXX.XXX.14.4.49744 > XXX.XXX.14.85.22: Flags [S], seq 1150830320, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 3381389031 ecr 0,sackOK,eol], length 0

• I've checked that launchd is indeed listening on port 22 by running sudo lsof -i -n -P | grep LISTEN. The output is similar to that on other machines where ssh works fine.
• I am able to access the server via vnc using the Screensharing app.
• I am able to ssh into another machine that's on the same local network, so it's not a firewall issue (and, as already mentioned, I do see the incoming requests on port 22).
• When trying to ssh with the -vvv flag, I get the following

% ssh -vvv user@hostname
    OpenSSH_9.3p2, LibreSSL 3.3.6
debug1: Reading configuration data /Users/xxx/.ssh/config
debug1: /Users/xxx/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/xxx/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/xxx/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to hostname port 22.

At this point I've run out of ideas of things to try to debug the issue... Any ideas?

Answer 1

Assuming the date and time on both computers are within 10 seconds of each other, you might need to look at networking when you get a "debug1: Connecting to hostname port 22." message.

If you manage the networks / IP ranges - perhaps there's something unusual or local you can look at with the switch / hub / router. If you don't have access to networking information or the switch - then a tcpdump to filter for that remote address / host might give one of us enough clues to help further. If you have both machines and they are close - setting up a point to point network would let you know if this is truly on the OS side of the other ssh side. (Although you've already looked at the tcpdump - so posting that can be problematic and many people won't parse them online)

I've never needed to enable logs / look at them on the macOS sshd side but perhaps that would also let you know more?

worst case you edit the post to explain a bit more:

• what OS version is ssh?
• What OS version is sshd?
• How exactly are things connected?
• Is the failure bilateral (can the problematic Mac ssh out but not allow ssh in)?