Watching the Watchers: On Its Own, AI Isn’t Watching, Or Thinking

Image: Citizen website.

Joseph Cox of 404 Media reminds us of three things that we know to be true about the new era of generative artificial intelligence:

1. AI isn’t a substitute for people.
2. AI isn’t a substitute for people.
3. AI isn’t… well, you get the picture.

As we’ve written before, AI works best when there’s a human in the loop. Take the case of Citizen.com, whose app is increasingly taking an AI-only approach to crime fighting. Because, really, what could possibly go wrong?

Plenty, as you can imagine. Without further ado, here’s 404 Media’s report on what happens when AI is left to its own devices, Citizen-style. It is prone to:

• Mistranslating “motor vehicle accident” as “murder vehicle accident.”

 

• Misinterpreting addresses.

 

• Publishing incorrect locations.

 

• Adding gory or sensitive details that violate Citizen’s guidelines.

 

• Sending notifications about police officers spotting a stolen vehicle or homicide suspect, potentially putting operations at risk.

 

• Writing alerts as if officers had already arrived on the scene, when in fact the dispatcher was only providing supplemental information while officers were en route.

 

• Duplicating incidents, failing to recognize that two pieces of dispatch audio are related to the same singular event.

 

• This was especially common with police chases, where dispatch continually provided new addresses. The “AI would just go nuts and enter something at every address it would get and we would sometimes have 5-10 incidents clustered on the app that all pertain to the same thing,” one source said.

 

• Omitting important details, such as whether a person was armed with a weapon.

​
The stakes are as strategic as they are tactical. One of Cox’s sources told him, “This could skew the perception of crime in a particular area,” as AI-created incidents proliferated.
 
By the way, the original name of Citizen – both the app and the company – was, perhaps tellingly, Vigilante. But that’s a story for another day.

DONATE & HELP US PROTECT YOUR PRIVACY RIGHTS