After several hours of looking and using
fs_usage to trace files, I have come to the conclusion that the certificates are indeed stored in the keychain. (which in hindsight is a DUH moment).
Here are the files iTunes reads on launch and during a sync on my mac.
18:38:16.776 stat64 /Library/Keychains/System.keychain 0.000010 iTunes
18:38:16.776 open F=5 (R_____) /Users/me/Library/Keychains/login.keychain 0.000007 iTunes
18:38:16.801 open F=5 (R_____) /Library/Keychains/System.keychain 0.000016 iTunes
18:38:26.013 open F=48 (R_____) /System/Library/Keychains/SystemRootCertificates.keychain 0.000024 iTunes
I have been unable to uncover which specific keys or chain of keys are needed. They keychain in my home folder appears to not be needed since other user accounts on my mac see content of my locked iPhone even before iTunes launches. Whether
usbmuxd or another low level daemon is reading a system level certificate or the API to access iOS data relies on a common library for access in bypassing the passcode lock on iOS 4 is unclear.
It is clear that the iOS passcode won't protect from someone that can access the files on your hard drive (and has the detailed knowledge that isn't yet clear which certificate or file has the keys stored)
FileVault won't protect you since the certificate appears to be stored outside the user folder.