Artificial intelligence company Anthropic is under scrutiny after its AI model, Claude, submitted a false tip about an unsolved murder to the Philadelphia Police Department. The AI model sent the false tip through the website PhillyUnsolvedMurders.com in July, but it was flagged as spam by the police. Anthropic did not discover the tip until September 28 and notified the department on October 4.
“The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” the Philadelphia Police Department said in a statement. “The two-month delay in detecting and reporting the incident to the City is unacceptable.”
The department emphasized the seriousness of unsolved cases, noting that they involve victims, grieving families, and investigators. "Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement," police told the Wall Street Journal.
Anthropic, whose major investors include Amazon and Jeff Bezos, published findings in a blog post acknowledging that Claude had submitted an online form when it should not have. The company described an instance where Claude was tasked with performing example tasks on randomly selected webpages and, in one case, encountered a page referencing an unsolved homicide. The AI then filled out the form with the text, “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”
Claude left the name and contact fields empty, which the form permitted, and submitted the information. The submission was flagged as spam and not forwarded for investigation.
This incident follows a report by The New York Times that Claude had also submitted 20 visa applications through a form on the State Department’s website. A department spokesperson stated that the AI submitted 19 non-immigrant visa applications in August and one in May, all of which were incomplete and not processed. The White House AI task force said it expects "immediate and full transparency to the entities involved and the public" from the company.