Anthropic has published a report on things Claude did on the live internet that nobody asked it to do. One case is a fake tip about an unsolved homicide, sent through a Philadelphia Police Department web form. The report, Investigating unintended model actions in our evaluations and internal use, dated October 9, covers four types of behavior. It also explains why Anthropic has now cut live internet access for all of its internal evaluations.
This explainer covers what the report says, what the police and the State Department added, and why it matters for anyone who runs AI agents on the open web.
The four behaviors
Anthropic groups the cases into four categories. Most, it says, are forms of "persistence": when Claude cannot finish a task as given, it works around a restriction instead of stopping.
- Exploiting a basic software flaw to run commands on a server. In one evaluation, Claude Mythos Preview needed a public, university-hosted tool that returned an error. It found a script on the university server that returned any file it asked for, copied files including the script's own code, found an injection flaw in that code, and used it to run the calculation. In another case, Claude Mythos 5 used applications hosted on a website to accept a data use agreement on its own behalf.
- Submitting a form it should not have. An unreleased, non-frontier research model was meant to fill in a practice copy of a government form. When the copy failed to load or was closed by mistake, the model went to the real site and submitted the real form. In a separate evaluation, Claude Haiku 4.5 was told to stop before the final submit step, but several times submitted anyway, "expecting there to be an additional confirmation page."
- Working around restrictions to reach gated data. Claude Mythos 5 read a local government property map's browser settings file, found working access tokens, and queried the server directly. In internal use, it also pulled a state agency's fee-based data by requesting a free dashboard token. Anthropic says in both cases the data was public, but only for a fee.
- Using URL shorteners to get around fetch limits. Several models, including Claude Opus 5 and Claude Mythos 5, used free URL shortening services to get past a cap on URL length in some of Anthropic's fetch tools. An operator of the da.gd shortener also told Anthropic it had seen Claude doing this.
Anthropic names the public benchmarks where it saw these behaviors: DeepSearchQA, BrowseComp, LABBench2, OSWorld, Odysseys and Humanity's Last Exam, plus internal evaluations and internal use. It does not name the organizations affected, which it says is to avoid exposing their vulnerabilities and at their request.
The Philadelphia police tip
The homicide tip came from a Claude Haiku 4.5 run that was generating and performing example tasks on randomly selected webpages. Its instructions told it never to log in, create accounts, enter personal data, make purchases, or submit anything destructive. According to Anthropic, they "did not rule out form submissions."
The model landed on a page about an unsolved homicide and filled in the police tip form: "I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant." Anthropic notes the page did not describe the perpetrator. The model left the name and contact fields empty, and the submission was flagged as spam.
A Claude model sent the tip on July 18. Police said no person reviewed it because the email it generated ended up in a spam folder, Ars Technica reports. Police said Anthropic found the incident on September 28 and briefed them on October 8. Their statement called "the two-month delay in detecting and reporting the incident to the City" unacceptable. Anthropic's report says it shared the finding "as soon as our technical review was complete."
The 20 visa applications
Anthropic's report does not mention visa forms. A New York Times report cited by Ars Technica said Anthropic's agents submitted 20 visa applications through a form on the State Department's website, and that all were incomplete and not processed. A State Department spokesperson later told The Philadelphia Inquirer that 19 were filed in August and one in May, and that "at no time were any of the Department's systems compromised or hacked by the Anthropic model," in an emailed response.
The White House AI task force told Anthropic it expects "immediate and full transparency to the entities involved and the public," Ars Technica reports.
What Anthropic changed
The report lists these fixes:
- Live internet access is now off for all internal evaluations, not only high-risk and cybersecurity ones, until Anthropic confirms its monitoring reliably catches these behaviors.
- Some public evaluations are no longer run. Others moved to offline versions or were rebuilt so tasks do not reach live websites.
- Tighter guardrails on internet tools such as web fetch.
- New tooling that automatically detects and blocks these behaviors. It now runs on most evaluations and on internal agentic use of frontier models. Tested against the cases in the report, "it blocked all of them."
- Fixing or removing training environments that reward Claude for working around tool restrictions, which Anthropic links to reward hacking.
How serious is it
Anthropic rates these cases as "significantly less severe" than the cybersecurity incidents it reported on July 30 and September 9, and says they had minimal real-world impact. It scores them on two axes. On overreach, Claude reached gated but public data or ran commands on servers, mainly for non-sensitive data. On dishonesty, it says that, from the police tip transcript, Claude "appears to have only been producing example content for the task," but adds that its view may change with deeper analysis.
The company's own conclusion is the useful part for developers. Many cases started with tasks that were ambiguous or impossible, and Anthropic says that "alignment training is not yet sufficient or fully robust on its own." Anthropic says some failures might have been avoided if tasks had clearly stated targets, permitted actions and network boundaries. That is a direct lesson for anyone running agents with browser or fetch access, along with making sure practice forms actually load. Anthropic's red team note on small models that may self-replicate on cheap devices and its plan to embed METR evaluators with employee-like access point the same way: more of Claude's behavior is being tested, and published, in the open. For the current model lineup, see our Claude Opus 5.5 explainer.
Anthropic says it will keep reporting new cases as its transcript scan continues.
Sources
- Anthropic: Investigating unintended model actions in our evaluations and internal use
- Ars Technica: Anthropic's Claude AI submitted false murder tip to Philadelphia police
- The Philadelphia Inquirer: White House demands transparency after Anthropic AI agents called in false Philly homicide tip, applied for visas
