Maybe because it’s working with Messages to block a photo if someone sends it to you.
There are some articles on the methods used and it appears it would be very simple for someone to create a very innocuous picture/photo that would actually trigger the alert mechanism. If, say, someone wanted to “get” someone.
This one just seems to be pretty filled will unintended consequences.
It’s very difficult to create a file with a given SHA256 hash, the algorithm has not been cracked yet and I’m not aware of any collision attacks to make it cheaper to do, so you’re looking at an enormous brute force requirement.
What’s the current US law about a file hosting service hosting illegal images unwittingly?
This seems to me they are blocking the upload itself, looking at the hash the phone sends before uploading the photo itself.
It’s certainly a slippery slope, but a hash of an existing photo known to be illegal is really the less intrusive way of being this extremely intrusive. The iMessage thing can be equally borderline if it only sends hashes and the hashes sent are compared and discarded and the photos themselves never stored unencrypted.
But I see the problem can arise mostly with political photos/memes. That would be an easy way of identifying opposition supporters and it’s an open door to abuse if mismanaged.
If it’s user-generated content and they respond to a takedown order, the hosting provider is protected unless it’s an ad for prostitution. This is the famous section 230 of the CDA and the infamous more recent FOSTA act.
If they are scanning hashes, it should be super easy to slightly modify any illegal photo to generate a new hash that doesn’t match the database.
Who are the poor people building the training data set? How does a new photo get registered in the db?
I used to work for the government. One project had me in the room full of people they hired to scan for illegal pornography. They only hired young women for the job. They were all extremely professional and took it very seriously. However, they tended to leave the role after a year or two. The entire process was as disturbing and alarming as you might imagine.
This isn’t true afaik. There’s a process called PhotoDNA that they use for this.
https://www.microsoft.com/en-us/photodna
The algorithm is smart enough to detect similar images despite attempts at alteration. Hence the human part of the check. There can be false positives, but it would be rare.
Yes, but this apple thing is supposedly only using hashes. So modifying a photo even slightly would indeed defeat it. Of course, who’s going to actually do that? And if the modified photos are distributed those hashes would be added to the DB also.
I’d be interested in seeing a sketch of the algo that generates matching hashes for different images (that have been modified). I’ll have to dig deeper into photoDNA.
PhotoDNA can identify a specific image regardless of its binary data. This is made possible because PhotoDNA looks at the visual content of the image, instead of exact binary image data. By calculating the mathematical distance (the Euclidean distance) between two PhotoDNA hashes it is possible to verify that the two hash values represent two different versions of the same image. Even if an image is edited very slightly, such as resized or saved in a different format, PhotoDNA will still match against that image.
I haven’t thought of hashes as something that carried information such that you could compute distances. I guess there’s some unusual hashing going on to make that happen.
They don’t, they’re deliberately obfuscating how it works for obvious reasons. My guess is it’s actually pretty standard ML.
It’s a nonstandard hash technique. Pretty neat tech.
“I should be working but this might be something close to something I can leverage”
In particular the actual hash algo here: pHash.org: Home of pHash, the open source perceptual hash library
That makes more sense. Normal hashes would be really use to fool and somewhat prone to false positives.
Normal hashes are basically mathematically immune to false positives in the reasonable lifetime of the universe, unless they’re flawed and a collision attack is discovered like md5. And even then you’d need to deliberately try to duplicate the hash.
Depends on the number of things you’re hashing. The more you hash, the closer you get to collisions, due to the pigeonhole principle.
So, Apple is only looking at hashes. But the hashes have enough information to reconstruct a fair amount of the image. This deal is getting worse all the time.
I would like to know how you reconstruct a fair amount of an image with a hash.
I’m not an expert in this so I may be missing something, but my comment was based off this: (pHash.org: Home of pHash, the open source perceptual hash library)
e.g.
In a normal hash, there isn’t a way to go from the hash to the source image, or from the hash to a similar image. Even if you have a single bit difference in the image you’ll get an entirely different hash.
In the 10x longer perceptual “hash”, they’re encoding key information about the image into the hash. It’s similar to (but a much more extreme version) of how a .mp3 can encode a much large .flac file. Again, not an expert, but I believe you can go from the perceptual hash back to a representation of the source image.
Yes, but the numbers are staggering. Stolen from stackoverflow (while hoping adapting the formatting works; how is math still hard to html?)
If we have a “perfect” hash function with output size n , and we have p messages to hash (individual message length is not important), then probability of collision is about (p^2)/(2^(n+1)) (this is an approximation which is valid for “small” p , i.e. substantially smaller than 2^(n/2) ). For instance, with SHA-256 ( n=256 ) and one billion messages ( p=10^9 ) then the probability is about 4.3x10^-60 .
A mass-murderer space rock happens about once every 30 million years on average. This leads to a probability of such an event occurring in the next second to about 10^-15 . That’s 45 orders of magnitude more probable than the SHA-256 collision. Briefly stated, if you find SHA-256 collisions scary then your priorities are wrong.
In a security setup, where an attacker gets to choose the messages which will be hashed, then the attacker may use substantially more than a billion messages; however, you will find that the attacker’s success probability will still be vanishingly small. That’s the whole point of using a hash function with a 256-bit output: so that risks of collision can be neglected.
Of course, all of the above assumes that SHA-256 is a “perfect” hash function, which is far from being proven. Still, SHA-256 seems quite robust.
Which goes out the window with a “smart” hash, but then you can’t know/prove they’re “the same” without looking at it. Which also makes the system vulnerable to DoS of the verifiers once someone knows the algorithm.
I’m sure it will work better than content id.