From your question it sounds like you can ssh into the machine but not connect via Finder. This leads me to think this is a File Sharing setting, not a Remote Login setting as the two are independent. So my answer applies to File Sharing settings.
If you go to System Preferences, then Sharing and for the File Sharing service, choose Options...
By default Mavericks has Share files and folders using SMB and Share files and folders using AFP selected but no users selected under Windows File Sharing.
It also appears (in my testing) that with these selections, when you try to connect to another computer listed in the Finder sidebar, it attempts to connect using SMB and does not fall back to trying AFP if the SMB connection fails.
The answer I found was to either
a. Specify a user to connect with using SMB in the options... section of File Sharing in System Preferences
b. Specify sharing over AFP only, deselecting SMB sharing
c. In Finder menu select Go then Connect to server... and then in the Server Address: field enter something like
afp://sharedcomputername.local
where sharedcomputername
is the name of the computer you're trying to connect to.