Short answer: Yes, choose "Show Invisible Items" in Keychain Access
(Originally, this answer incorrectly said: "There doesn't currently seem to be a way to recover the passphrase." The rest of the answer comparing how SSH key passphrases are stored in keychains on older and newer versions of macOS is kept here for reference.)
In Mac OS X 10.11 and earlier, SSH key passphrases were stored by ssh and ssh-add in the user's login keychain, using the following properties (pseudocode adapted from Apple OpenSSH-195.40.1):
{
kSecClass: kSecClassGenericPassword,
kSecAttrAccount: pathToPrivateKey,
kSecAttrLabel: "SSH: " + pathToPrivateKey,
kSecAttrService: "SSH"
}
As of macOS 10.12, the passphrase is queried and stored with properties that force it to be stored in the "syncable" keychain (the database-backed "Local Items", also called "iCloud Keychain" if that is enabled), with access controlled by an "Access Group"; ssh will ignore the old items in your login keychain.
From Apple OpenSSH 209:
{
kSecClass: kSecClassGenericPassword,
kSecAttrAccount: pathToPrivateKey,
kSecAttrLabel: "SSH: " + pathToPrivateKey,
kSecAttrService: "OpenSSH",
kSecAttrNoLegacy: TRUE,
kSecAttrAccessGroup: "com.apple.ssh.passphrases"
}
Apple has not enabled the security command-line tool to access the contents of the syncable keychain, and by default (unless you select "Show Invisible Items") Keychain Access doesn't show the complete contents of it either, even though the ssh keychain item is not explicitly marked as invisible.
If you wanted to try to obtain the password without using Keychain Access, I don't think it would be straightforward or maybe even possible to write a program that fetches the passphrase from the keychain, since access to it is controlled by Apple's code signing.
You can see some limited details about the passphrases stored in your syncable keychain using the sqlite3 command, but most of the interesting fields like the item labels are encrypted.
$ sqlite3 Library/Keychains/*/keychain-2.db "select rowid, datetime(mdat+978307200, 'unixepoch', 'localtime'), agrp from genp where agrp = 'com.apple.ssh.passphrases' order by mdat asc"
45|2017-01-07 13:53:13|com.apple.ssh.passphrases
46|2017-01-08 23:59:25|com.apple.ssh.passphrases
