You likely have a shell quoting problem. The command works just fine for me in 10.11.4:
> which openssl
/usr/bin/openssl
> openssl version
OpenSSL 0.9.8zh 14 Jan 2016
> openssl enc -aes-256-cbc -salt -in file.zip -out file.zip.enc; echo $?
enter aes-256-cbc encryption password:
Verifying - enter aes-256-cbc encryption password:
0
> openssl enc -aes-256-cbc -d -in file.zip.enc -out file.zip; echo $?
enter aes-256-cbc decryption password:
0
Check to make sure that your calling program is treating -aes-256-cbc
as a single option and isn't quoting it when sending it in to the shell. It looks like it's calling the program with option cluster "-aes-256-cbc -in file.enc"
which actually isn't a valid, single option for openssl
.
If you can provide more details about how you're calling openssl
I can give you more specific guidance.