Apple is said to be close to announcing a new photo identification features that will use hashing algorithms to match the content of photos in user’s photo libraries with known child abuse materials, such as child pornography.
Apple’s system will happen on the client — on the user’s device — in the name of privacy, so the iPhone would download a set of fingerprints representing illegal content and then check each photo in the user’s camera roll against that list. Presumably, any matches would then be reported for human review.
Apple has previously said it employs hashing techniques as photos are uploaded to iCloud. This new system would be done on the client side, on the user’s device. Apple is yet to officially announce this new initiative, and the details will matter.
Cryptography and security expert Matthew Green notes that the implications of such a rollout are complicated. Hashing algorithms are not foolproof and may turn up false positives. If Apple allows governments to control the fingerprint content database, then perhaps they could use the system to detect images of things other than clearly illegal child content, such as to suppress political activism.
However, note that all photos uploaded to iCloud Photos for backup and sync are not stored end-to-encrypted anyway. Photos are stored in an encrypted form on Apple’s server farms, but the keys to decrypt are also owned by Apple. This means that law enforcement agencies can subpoena Apple and see all of a user’s uploaded photos.
MacDailyNews Take: This sounds great at first glance (detecting and rooting out purveyors of child pornography) and horrible once you think about it for more than a second (massive potential for misuse).
It’s a huge can of worms. Apple’s implementation of this new photo identification feature will be crucial.
I’ve had independent confirmation from multiple people that Apple is releasing a client-side tool for CSAM scanning tomorrow. This is a really bad idea.
— Matthew Green (@matthew_d_green) August 4, 2021
Initially I understand this will be used to perform client side scanning for cloud-stored photos. Eventually it could be a key ingredient in adding surveillance to encrypted messaging systems.
— Matthew Green (@matthew_d_green) August 5, 2021
The ability to add scanning systems like this to E2E messaging systems has been a major “ask” by law enforcement the world over. Here’s an open letter signed by former AG William Barr and other western governments. https://t.co/mKdAlaDSts
— Matthew Green (@matthew_d_green) August 5, 2021
This sort of tool can be a boon for finding child pornography in people’s phones. But imagine what it could do in the hands of an authoritarian government? https://t.co/nB8S6hmLE3
— Matthew Green (@matthew_d_green) August 5, 2021
The way Apple is doing this launch, they’re going to start with non-E2E photos that people have already shared with the cloud. So it doesn’t “hurt” anyone’s privacy.
But you have to ask why anyone would develop a system like this if scanning E2E photos wasn’t the goal.
— Matthew Green (@matthew_d_green) August 5, 2021
But even if you believe Apple won’t allow these tools to be misused 🤞there’s still a lot to be concerned about. These systems rely on a database of “problematic media hashes” that you, as a consumer, can’t review.
— Matthew Green (@matthew_d_green) August 5, 2021
The idea that Apple is a “privacy” company has bought them a lot of good press. But it’s important to remember that this is the same company that won’t encrypt your iCloud backups because the FBI put pressure on them. https://t.co/tylofPfV13
— Matthew Green (@matthew_d_green) August 5, 2021
The theory is that you will trust Apple to only include really bad images. Say, images curated by the National Center for Missing and Exploited Children (NCMEC). You’d better trust them, because trust is all you have.
— Matthew Green (@matthew_d_green) August 5, 2021
But there are worse things than worrying about Apple being malicious. I mentioned that these perceptual hash functions were “imprecise”. This is on purpose. They’re designed to find images that look like the bad images, even if they’ve been resized, compressed, etc.
— Matthew Green (@matthew_d_green) August 5, 2021
This means that, depending on how they work, it might be possible for someone to make problematic images that “match” entirely harmless images. Like political images shared by persecuted groups. These harmless images would be reported to the provider.
— Matthew Green (@matthew_d_green) August 5, 2021
Regardless of what Apple’s long term plans are, they’ve sent a very clear signal. In their (very influential) opinion, it is safe to build systems that scan users’ phones for prohibited content.
That’s the message they’re sending to governments, competing services, China, you.
— Matthew Green (@matthew_d_green) August 5, 2021
Whether they turn out to be right or wrong on that point hardly matters. This will break the dam — governments will demand it from everyone.
And by the time we find out it was a mistake, it will be way too late.
— Matthew Green (@matthew_d_green) August 5, 2021