The American Civil Liberties Union of Northern California said Thursday that in its new test of Amazon’s facial recognition system known as Rekognition, the software erroneously identified 28 members of Congress as people who have been arrested for a crime.
According to Jake Snow, an ACLU attorney, the organization downloaded 25,000 mugshots from what he described as a “public source.
The ACLU then ran the official photos of all 535 members of Congress through Rekognition, asking it to match them up with any of the mugshots—which ended up being 28.
Of those 28, the ACLU’s test flagged six members of the Congressional Black Caucus, including Rep. John Lewis (D-Georgia), a legendary African-American civil rights activist.
Facial recognition historically has resulted in more false positives for African-Americans.
As Ars has reported before: if the training data is heavily skewed toward white men, the resulting recognizer may be great at identifying other white men but useless at recognizing anyone outside that particular demographic.
“We used the default level of confidence that Amazon uses, its 80 similarity score,” Snow told Ars.
This test comes just two months after the Congressional Black Caucus wrote to Amazon CEO Jeff Bezos, expressing concern over the “profound negative consequences” of the use of such technology.
“And running the entire test cost us $12.33—less than a large pizza,” Snow added in the Thursday blog post.
The ACLU is concerned that over-reliance on faulty facial recognition scans, particular against citizens of color would result in a possible fatal interaction with law enforcement. Amazon’s Rekognition has already been used by a handful of law enforcement agencies nationwide.
Because of these substantive errors, Snow said the ACLU as a whole is again calling on Congress to “enact a moratorium on law enforcement’s use of facial recognition.”
When Ars contacted Amazon, Nina Lindsey, a company spokeswoman, would only provide a corporate statement saying that the ACLU should have increased the confidence threshold. The Orlando Police Department ended its test just last month.
“While 80 percent confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn’t be appropriate for identifying individuals with a reasonable level of certainty,” the company said. “When using facial recognition for law enforcement activities, we guide customers to set a higher threshold of at least 95 or higher.”
Lindsey would not respond to Ars’ questions as to how many law enforcement agencies use Rekognition at present, and whether those agencies abide by Amazon’s 95 confidence recommendation.
“Finally, it is worth noting that in real world scenarios, Amazon Rekognition is almost exclusively used to help narrow the field and allow humans to expeditiously review and consider options using their judgement (and not to make fully autonomous decisions), where it can help find lost children, restrict human trafficking, or prevent crimes,” the statement concluded.
