2

Something in macOS Catalina (10.15.1) is interfering with ssh port forwarding (needed for localhost debugging and developing against a web server system deployed in AWS).

I assume something with the enhanced security features, but I've been unable to figure out what from searching and poking around.

The tunnel seems to start up fine, but when a client tries to connect the connection is denied. The tunnel & app startup commands are the same as what was working fine in all previous releases of the os, afaik, but are failing on this new 10.15 install (nothing ported, fresh setup of tools. I did change my default shell back to bash.)

Network configuration: There is a redis server in an AWS VPC that's not publicly routable. We have a bastion server that is publically routable that can forward the connection via ssh. I can connect to the bastion server for an interactive session, so the keys and passphrases are all ok as far as I know. There's no VPN to AWS, so the bastion host is the only way into the private subnets.

Here's the command we use to start up the tunnel (prodbastion is defined in .ssh/config):

ssh -v -N -L 6399:redis.<HOSTNAME>:6379 prodbastion

This logs verbosely in the shell, here's the end of the supposedly successful authentication startup, followed by the repeated client connection failures


debug1: Authentication succeeded (publickey).
Authenticated to <bastionprod hostname> ([<bastion prod ip>]:22).
debug1: Local connections to LOCALHOST:6399 forwarded to remote address redis.<HOSTNAME>:6379
debug1: Local forwarding listening on ::1 port 6399.
debug1: channel 0: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 6399.
debug1: channel 1: new [port listener]
...
debug1: channel 2: free: direct-tcpip: listening port 6399 for redis.<HOSTNAME> port 6379, connect from 127.0.0.1 port 51454 to 127.0.0.1 port 6399, nchannels 3
debug1: Connection to port 6399 forwarding to redis.<HOSTNAME> port 6379 requested.
debug1: channel 2: new [direct-tcpip]
channel 2: open failed: administratively prohibited: open failed
debug1: channel 2: free: direct-tcpip: listening port 6399 for redis.<HOSTNAME> port 6379, connect from 127.0.0.1 port 51457 to 127.0.0.1 port 6399, nchannels 3
debug1: Connection to port 6399 forwarding to redis.<HOSTNAME> port 6379 requested.
debug1: channel 2: new [direct-tcpip]
...

The client is a nodejs web app, needing data from redis. the logs there just say more or less cannot connect:

2019-12-09T17:13:51.286Z - info: Client Redis connection: {"host":"localhost","port":"6399","database":"4"}
2019-12-09T17:13:51.299Z - info: mongoose connected
2019-12-09T17:13:51.332Z - error: Redis error occured: AbortError: Ready check failed: Redis connection lost and command aborted. It might have been processed.
2019-12-09T17:13:51.473Z - error: Redis error occured: AbortError: Ready check failed: Redis connection lost and command aborted. It might have been processed.
...

It seems to have the same failure with the system ssh or openssh installed with brew.

Update in response to questions:

  • It was my error all along, the hostname in the tunnel command didn't exist but my bash history led me to believe it did
  • firewall was off
  • telnet could not connect to 6399 with the tunnel up to a bad hostname
  • telnet (and our redis-using app) could connect with the right hostname.
  • argh and sorry about the confusion!
  • Can you please run telnet 127.0.0.1 6399 (after the tunnel is made) and post the response? Note, you may need to install telnet if it is not already installed. – JBis Dec 16 '19 at 4:10
  • 1
    Can you also confirm that the Firewall (System Preferences > Security & Privacy > Firewall) is off? – JBis Dec 16 '19 at 4:15
  • Can you also confirm that you are able to do this from another computer so we know the server isn't the issue? – JBis Dec 16 '19 at 4:17
  • FYI, I have been able to do this exact configuration (except instead of Redis I was using MySQL) on Catalina so it is definitively possible. – JBis Dec 16 '19 at 4:23
3
+50

From What's ssh port forwarding and what's the difference between ssh local and remote port forwarding:

I have drawn some sketches ssh tunnel starting from local


ssh tunnel starting from remote

Introduction

  1. local: -L Specifies that the given port on the local (client) host is to be forwarded to the given host and port on the remote side.

    ssh -L sourcePort:forwardToHost:onPort connectToHost means: connect with ssh to connectToHost, and forward all connection attempts to the local sourcePort to port onPort on the machine called forwardToHost, which can be reached from the connectToHost machine.

  2. remote: -R Specifies that the given port on the remote (server) host is to be forwarded to the given host and port on the local side.

    ssh -R sourcePort:forwardToHost:onPort connectToHost means: connect with ssh to connectToHost, and forward all connection attempts to the remote sourcePort to port onPort on the machine called forwardToHost, which can be reached from your local machine.

Examples

Example for 1

ssh -L 8080:localhost:80 SUPERSERVER You specify that a connection made to the local port 80 is to be forwarded to port 8080 on

SUPERSERVER. That means if someone connects to your computer with a webbrowser, he gets the response of the webserver running on SUPERSERVER. You, on your local machine, have no webserver running.

Example for 2

ssh -R 80:localhost:8080 tinyserver You specify, that a connection made to the port 80 of tinyserver is to be forwarded to port 8080 on

your local machine. That means if someone connects to the small and slow server with a webbrowser, he gets the response of the webserver running on your local machine. The tinyserver, which has not enough diskspace for the big website, has no webserver running. But people connecting to tinyserver think so.

More examples

Other things could be: The powerful machine has five webservers running on five different ports. If a user connects to one of the five tinyservers at port 80 with his webbrowser, the request is redirected to the corresponding webserver running on the powerful machine. That would be

ssh -R 80:localhost:30180 tinyserver1
ssh -R 80:localhost:30280 tinyserver2
etc.

Or maybe your machine is only the connection between the powerful and the small servers. Then it would be (for one of the tinyservers that play to have their own webservers):

ssh -R 80:SUPERSERVER:30180 tinyserver1
ssh -R 80:SUPERSERVER:30280 tinyserver2
etc
|improve this answer|||||

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .