According to the Federal Criminal Police Office (BKA), 99,375 of the 205,728 reports forwarded by the US-based National Center for Missing and Exploited Children (NCMEC) were not criminally relevant, an error rate of 48.3%. This is a rise from 2023, when the number of false positives already stood at 90,950.
Indeed 50% false positive rate sounds surprisingly good, but this is under the "voluntary scheme" where Meta/Google/MS etc are not obligated to report. Notably missing from the article is the total number of scanned messages to get down to 200k reports. To my knowledge, since it's voluntary, they can also report only the very highest confidence detections. If the Danish regime were to impose reporting quotas the total number of reports would rise. And of course -- these are reports, not actually convictions.
Presumably the actual number of criminals caught by this would remain constant, so the FP rate would increase. Unless of course, the definition of criminal expands to keep the FP rate low...
I recall a half decade back, there was discussion of the quit rate of employees, maybe Facebook?, due to literal mental trauma from having to look at and validate pedophile flagged images.
Understand there is pedophilia, then there's horribly violent, next level abusive pedophilia.
I used to work in a department where, adjacently, the RCMP were doing the same. They couldn't handle it, and were constantly resigning. The violence associated with some of the videos and images is what really got them.
The worst part is, the more empathetic you are, the more it hurts to work in this area.
It seems to me that without this sad and damaging problem fixed, monitoring chats won't help much.
How many good people, will we laden with trama, literally waking up screaming at night? It's why the RCMP officers were resigning.
I can't imagine being a jury member at such a case.
Because of this issue, many departments put in much stricter protocols for dealing with this kind of material. Only certain people would be exposed to classify/tag it, and these people would only hold that post of a limited period of time. The burden on those people doesn't change, but it can be diluted to mitigate it somewhat.
Its a real and sad problem, but not one that I think can be fixed with technology. To much is on the line to allow for a false positive from a hallucinating robot to destroy a person(s) life.
Presumably the actual number of criminals caught by this would remain constant, so the FP rate would increase. Unless of course, the definition of criminal expands to keep the FP rate low...