Claude AI sent Philadelphia police phony murder; Anthropic argues incident 'less severe' than other cybersecurity episodes

An alarming artificial intelligence failure has raised fresh questions about the risks posed by increasingly autonomous AI agents. An Anthropic model submitted a fabricated tip about an unsolved homicide to Philadelphia police. Authorities have criticised the company because the incident was reported to the police two months after it occurred.

The Philadelphia Police Department revealed that the false tip was submitted on July 18 through PhillyUnsolvedMurders.com, a public website where people can share information about unsolved killings. The AI model presented itself as someone who might have knowledge of the case, according to the department.

Anthropic said the model was running a test that involved interacting with randomly selected websites when it reached the homicide website and submitted the false information.

The tip was flagged as spam and never reached the department's Real-Time Crime Centre for vetting, police said, adding that AI-generated misinformation involving real crimes cannot be treated lightly. Authorities added that there was no evidence that police systems had been breached or that departmental data had been compromised.

Anthropic criticised over two-month reporting delay

The Philadelphia police department said Anthropic discovered the incident on September 28, shut down the automated testing process responsible and introduced an additional validation step for future tests. However, the company waited till October 7, almost two months to notify the police.

“The two-month delay in detecting and reporting the incident to the City is unacceptable,” the Philadelphia Police Department said. They acknowledged that existing safeguards had limited the damage but stressed that the incident remained serious.

“[These safeguards] do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide,” the department said.

“Unsolved cases involve real victims, grieving families and investigators working to secure answers,” it added.

Anthropic meanwhile argued that the newly disclosed incident, which was one among many, "had minimal real-world impact" and was "significantly less severe" than other cybersecurity incidents previously reported.

Anthropic reveals other unintended AI actions

The incident was among several examples of unintended behaviour outlined in a report Anthropic published on Friday (Oct 09). The company said an internal review of its Claude model identified four broad categories of incidents: exploiting basic coding flaws, submitting forms on websites, bypassing token or fee requirements, and using short URLs to get around other limits.

The report also referred to interactions involving the White House and other US government agencies.

The company has temporarily disabled internet access for Claude during internal testing while it reviews its safeguards. The move is intended to ensure that security and monitoring measures can reliably detect similar behaviour before internet access is restored, according to the report.

Growing concerns over autonomous AI agents

The episode adds to mounting concerns about AI agents, systems designed to carry out multiple steps and interact with websites and digital services with limited human supervision. Previously, a rogue OpenAI agent had reportedly escaped its testing environment and breached systems at AI platform Hugging Face.