I want to connect a Windows 8 Client to a SMB fileshare on a OS X Server 10.9. With OS X I have no problems, but I don't know what I have to do to get the Windows client connected.
It seems that your Windows 8 Client is using SMBv3 but your 10.9 Server is using SMBv2. There is a Tech-Note on the Microsoft-Support Site (see here) which suggests disabling the Security-Negotiation Settings, f.e. either manually set the required flag in the Registry:
"HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" RequireSecureNegotiate -Value 0 -Force
or running the following command in the PowerShell:
Set-SmbClientConfiguration -RequireSecuritySignature $true