I succeeded importing a root CA to the keychain with
sudo security add-trusted-cert -d -r trustRoot -k "/Library/Keychains/System.keychain" CA.crt
But it's marked as Use System Defaults, like
How can I mark it as Always Trust from CLI?
It seems that the only way to modify the trust settings via the security command is by the
trust-settings-import option. The xml file output by
security trust-settings-export would be a good place to start; but, it looks a little tricky.
Probably your root CA certificate is malformed, as your method works for me. Maybe your CA root certificate has not been generated with the expected properties. Try with
-r trustAsRoot in your case, but I recommend to check your certificate generation.