Ask Different is a question and answer site for power users of Apple hardware and software. It only takes a minute to sign up.
Sign up to join this community
Siri must "listen" for commands, however, that being said if it does not receive the command prompt "Hey Siri," can users rely on the algorithm to not record/send conversations or other private audio?
If device specificity is required for the question. Assume an iPhone X with IOS 13.X
Your asked about the trust-worthiness of an algorithm. The answer to your question is a firm and absolute "No" - users can not rely on the algorithm to not record/send conversations or other private audio. Some algorithms are definitely better than others, but no algorithm is perfect - what is invented by the human mind can be defeated by the human mind.
Let's muse over another word in your question: rely. Reliance is not a binary state of mind; it's much more subtle than that. Consider that we may rely on the mail carrier to deliver our mail each day, but as the consequence of failure is not catastrophic, we do not rely upon it heavily. OTOH, if you are a diabetic living in a remote location, your reliance on the mail carrier's delivery of your insulin has far greater implications.
The "threat" is also a factor in our reliance. Security-wise, perhaps the thing we have going for us is that nobody really cares what we say or write or do. But if we happen to attract attention to ourselves - perhaps as a suspect, or the target of a criminal enterprise, or have become involved in "matters of national security" as this chap recently did, then our level of reliance on algorithms - and computer security in general - becomes a different matter. And so it is with your reliance on Apple's algorithm: does the mail carrier bring you junk, or does he/she deliver insulin?
Finally, let's consider the other variable in reliance: the quality of the algorithm. Reliance is not a binary state, and no algorithm is perfect. Nevertheless we must still consider how good it is in determining how heavily we should rely upon it.
Unfortunately it is a practical impossibility for us to know how secure Apple's algorithm is:
it's not open source - people outside Apple have to disassemble the code to inspect for flaws. They cannot read the source
even if it were open source, it wouldn't be perfect
even if it were perfect, Apple still makes mistakes
Apple could even abuse their privilege or not catch an employee that attempts this abuse
Instead, and as a substitute for a quantitative analysis of the algorithm itself, perhaps the best we can manage is an attempt at objective assessment of some facts. Consider the following:
A complete and massive failure of the algorithm would be bad for Apple's business. Consequently, it seems likely that Apple would put a serious effort into the algorithm, and safeguards for the data.
Apple has weighed up Siri's business value against the risks of failures of their algorithm, and have made the decision to retain Siri. Impose may be a better word than retain in this case as Apple has not provided an option to remove or un-install the Siri software.
These facts do not shed much light on the actual quality of Apple's algorithm, but they do speak to motives and motivation. We can safely conclude that Apple has reasonably strong financial incentives to design a reliable algorithm. However, it also seems safe to conclude that Apple considers our private data to be within their purview. Why else would they not provide an option for removal of Siri to accommodate their customers' preferences?
None of this is intended as "Apple-bashing". I am an Apple customer - I have spent more money than I'll ever admit on their products. But I feel it's also important to be realistic. Apple's interests do not coincide perfectly with ours. Should we rely on Apple's judgments and algorithms to protect and safeguard our security and privacy? I hope this answer has provided a little food for thought.
I have just read @Bradley's answer to this question. I have to say that if the allegations leveled by Apple's contractor in Ireland over the "Siri Incident" are true, then Apple deserves far more than the "Apple-bashing" I tried to avoid in my answer. They deserve substantial fines - perhaps even criminal proceedings. It is simply unacceptable that anyone or any organization be allowed to use their wealth and influence to trample the rights of others. More to the point of answering this question, this episode (if true) speaks volumes on how we should consider relying on Apple's software.
The answer is it is always listening. The recent article in the Guardian, of the former apple contractor, demonstrates precisely why the best option for you is to disable Siri all together if you want to mitigate the risk of yourself being recorded. Equivalently, buy the cheapest non-smartphone.
This is not just an Apple-centric issue. Unfortunately, when you accept the terms and conditions, you are giving Apple, and other companies the right to use your data however they want. This should not be the case when you are not using an application, but many times over it is the case.
We can only tell you what Apple publicly states. Can you rely on it, well that's entirely up to you. IMO, I trust Apple and Microsoft orders of magnitude more than I do Google, Facebook, Twitter, et al. That doesn't mean I trust them implicitly, either.
Specific to this question, it is addressed in this press release from Apple: Improving Siri’s privacy protections. Below is a quick summary of what's in the press release.
Yes.
Am I worried about it?
Not worried, exactly. I trust it, for now, but remain skeptical. Siri is listening, but if most of what Siri does is on device, there's no need to record/process your conversation until it hears "Hey Siri;" and responding to that has no need to be sent to servers for initial processing.
The German-language TV show Quer, broadcast by BR, the Bavarian public broadcaster, aired a segment back in March (prior to Covid-19-related restrictions in Bavaria) which can be found on Youtube (the auto-translated subtitles aren’t bad in my initial impression). About 1 minute in, they team up with the Chaos Computer Club to determine whether Siri sends data while listening without the trigger words Hey, Siri being added by putting themselves between the router and the internet to record every packet of data sent.
The positive control (a statement initiated with Hey Siri) immediately sends out data as supposed. Later in the segment (about 3:40) they perform a negative control and see absolutely no data usage. This negative control was designed to use a couple of trigger words (holiday, flight) that show the desire to purchase a non-everyday product making the speaker an incredibly interesting target for targeted advertising – one of the main criticisms that are being voiced.
Obviously, a 5-minute segment in a 45-minute show that covers a range of topics cannot cover all relevant aspects. Most notably, they only show checking for outgoing internet connections so any recordings kept locally on the phone would slip through the cracks. Furthermore, n = 1 for both positive control and experiment as of the broadcast segment which is obviously far below statistical significance. Nonetheless, this experiment provides an indication towards a No answer: at the very least it seems like Siri is not sending private communications to the outside world.
