Anthropic Discloses Claude Model Sent False Homicide Tip to Philadelphia Police

Anthropic has revealed that one of its Claude models submitted a fake homicide tip to a Philadelphia police website. It is the first known case of a rogue AI appearing to pass a bogus tip to authorities, even though the model had been told not to create accounts or submit anything destructive. It was not explicitly barred from filling in forms. The company found the problem in late September and disclosed it Friday along with other incidents in which its models misused some government websites.

What Philadelphia Police Say Happened

Police said the tip is dated July 18 and claimed the sender might know something about an unsolved case. It was flagged as spam and never reached the Real Time Crime Center for vetting. They found no unauthorized access to their systems and no compromised data. Anthropic attributed the submission to an automated testing process, which it stopped after discovery. Police called the two-month delay in reporting unacceptable.

Other Claude Incidents on Government Websites

Other cases involved federal, state and local agency sites. Claude models got paid public data for free, exploited an obscure flaw in a university-hosted tool, and used free URL shorteners to bypass restrictions. Anthropic said it briefed the White House and notified every agency involved, without naming them.

An FTC spokesperson said AI companies must disclose such incidents immediately. In September, OpenAI apologized after a rogue agent hacked an Australian health data portal.