In order for Apple to optimise the photos and videos on my devices, does it's cloud servers have access to the photo and video data? How else are they able to resize them? This is concerning for privacy reasons.
Apple clearly has access to the files, but my understanding is the files get encrypted on your device before they are uploaded.
I would start with https://support.apple.com/en-us/HT202303 where Apple says photos are encrypted in transit and in the cloud.
Whether you or Apple has a valid decryption key depends on date and time and iOS version and your settings on your account. Even though it's clear all data is encrypted while it is in transit and in the cloud for photos, they are likely decryptable by Apple alone without needing your password since you can browse them at https://iCloud.com and most browsers cannot handle decryption of that volume of data locally and securely. So, you should probably not upload anything to iCloud you don't encrypt yourself in a way you know Apple can't unlock if that's your main concern.
The optimization almost certainly happens on the device for two reasons.
- To allow Apple to not need the decryption key going forward (this is evolving as Apple reacts to governmental requests for data collection and unlocks).
- To allow Apple to not have to pay for the CPU bill to do the optimization.
I hypothesis (I have ver little evidence for this) that due to the encryption requirements for photos in the cloud, that Apple cannot access the photo data to perform the optimisation steps itself. So the device, first uploads an original copy, produces an optimised version including thumbnails, then deletes the original. Consumer devices (devices that didn't produce the original) then download the original version, produce an optimised copy for it's specifications, then deletes the original.
Also while on Wifi, the devices are capable of passing the originals directly to each other, rather than over the cloud, but I don’t know if they do.
The implications for this is that the process sacrifices network bandwidth for privacy, while a side plus being that the computation cost for doing this for all of Apple's millions of customers and their thousands of photos and videos is offload from their servers and onto the customer's various devices. Apple already explained that their data mining for the new improved event summery and slideshows in the new OS's (iOS 10 / macOS) will be done on device, not in the cloud.
I imagining this must be the case, unless Apple has some fancy photo / video format that allows the to resize the photo / frames somehow by handling the data in it's encrypted form. But that seems like magic to my CS understanding...