A computer picked the wrong man. Police arrested him

Now that I have your attention on the Flock Camera issue – I think it is worth revisiting an issue I’ve been writing about that should be just as chilling – if not more.

In January 2020, Detroit police arrested a man named Robert Williams on his front lawn, in front of his wife and his two little girls. He spent 30 hours in a dirty, crowded cell. He hadn’t done anything. A computer picked him.

A watch store had been robbed about a year and a half earlier. Police took a blurry frame from the security video and ran it through facial recognition software. The software matched it to an expired driver’s license photo of Williams.

He wasn’t even the software’s best guess – he was the ninth-best match. Nobody checked where he was that day before they arrested him.

Facial recognition is exactly what it sounds like. A computer scans a photo of a face and hunts through millions of other photos – mugshots, driver’s licenses, social media – looking for a match. Almost half the federal agencies with police on the payroll – 20 of 42 – use it in some form, according to a 2021 report from the federal government’s own watchdog.

And there’s no federal law that says how. No accuracy standard. No warrant requirement. No rule against a match being the whole case.

The software also gets Black faces wrong more than white ones.

The government’s own testing found that when software searches a mugshot database, Black women get falsely matched more often. At least seven people in this country have been arrested for crimes they didn’t commit because of a bad match. One was a Detroit woman who was eight months pregnant. Police held her for 11 hours on a carjacking charge. Nobody had described the carjacker as pregnant.

Williams sued. In 2024, Detroit paid him $300,000 and agreed to what the ACLU calls the strongest facial recognition rules of any police department in the country. Police there can’t arrest anyone on a software match alone anymore. That fix didn’t come from Congress. One man fought one city and won.

So where’s Memphis in all this?

MPD already runs an AI-powered command center downtown, with dozens of cameras and drones feeding it. When the police chief was asked about facial recognition in November 2024, she said the cameras are used “only for objects” and that “we are not using our technology at this time for that.”

At this time.

Memphis has no ordinance on police facial recognition – not a ban, not a rule, not even a reporting requirement. At least 16 American cities have one.

Meanwhile, reporters just uncovered a Homeland Security plan to hand facial recognition apps to local police who work with ICE – an app a lot like the one federal agents already carry, and it stores every scanned face for 15 years. And Memphis police are part of the Memphis Safe Task Force right now – 31 local, state and federal agencies, National Guard included. The only thing between those cameras and your face is a decision nobody has to ask your permission for.

I’ve been writing about this the whole campaign – the license plate cameras logging every car in Shelby County, the world’s largest supercomputer going up in our backyard, the AI report Congress ordered and then ignored, and a whole page of receipts on what’s true and false about data centers.

It’s all one story. The machines are being pointed at regular people, and nobody with power around here is writing the rules.

Here’s where you come in.

All over this district, people are already showing up to county commissions to fight data centers – over water, over power bills, over the turbine exhaust Boxtown has been breathing. Keep fighting those fights.

But we need to add this to the list, because what runs inside those buildings is the software that scans your face, reads your plate, and grades your life. When you petition your local government about a data center, petition them about data privacy in the same breath.

And be clear-eyed about where the fix comes first.

Congress should set a floor – a warrant and accuracy standards before police run a face search, and no arrest on a match alone. That’s the Detroit rules made national, and a House member can push for it.

But I won’t pretend a federal law is coming fast.

Congress’s own AI task force wrote a 250-page report and went home. The faster fix is local.

In Oakland, police have to bring surveillance tech purchases to elected officials in public before buying – a written use policy and annual reporting. San Francisco had the same rule until voters loosened it for police in 2024. The Memphis City Council could pass that on its own. Yes, the state legislature has overridden Memphis before. Fight anyway. Make them do it in public, on the record, in an election year.

Robert Williams was arrested in front of his daughters because no one checked the computer’s work. The rule we need is one sentence long: a computer’s guess is not probable cause.

Sources

Correction, September 27th, 2026: This essay first said about two-thirds of police agencies in this country use facial recognition, according to the government’s watchdog – the watchdog’s 2021 report counted 20 of 42 federal agencies with law enforcement officers. It said Homeland Security’s app for local police is the same one federal agents carry – reporters say it works a lot like it, but it isn’t clear it’s the same. And it said Memphis police work alongside 31 federal and state agencies in the Memphis Safe Task Force – the 31 include local agencies too. It also said the watch store was robbed two years before Robert Williams’s arrest – it was about a year and a half. It said San Francisco still makes police get approval before buying surveillance tech – voters loosened that rule for police in 2024. And it said Memphis could pass a rule like that in a single council vote – city ordinances take three readings.

Update, September 27th, 2026: A few lines now say who says so and when, including the date of the police chief’s comment. It also now says the government’s testing found Black women got more false matches in the kind of database search police run.

Everything I have written about AI is in one place at rachelandthecity.com/the-ai-files.

Sources

Share this: