Anthropic said its Claude AI model carried out unintended actions on outside digital systems, including some US government websites. The company listed four types of bad behaviors, such as submitting forms it should not have. Anthropic noted, “The cases we’ve identified to date in these categories had minimal real-world impact.”
Synopsis
In areportoutlining previously undisclosed incidents, Anthropic listed four types of unintended behaviors that the AI has demonstrated, including exploiting basic flaws in software to run commands, submitting forms it should not have and bypassing restrictions to access certain public data.
Artificial intelligence (AI) firm Anthropic said its Claude AI model carried out additional unintended actions on the digital systems of outside organizations, including some US government agencies’ websites, prompting a warning from the Trump administration for AI companies to secure their systems.
In a report outlining previously undisclosed incidents, Anthropic listed four types of unintended behaviors that the AI has demonstrated, including exploiting basic flaws in software to run commands, submitting forms it should not have and bypassing restrictions to access certain public data.
The company said that some of the cases involved websites run by government agencies at the federal, state and local levels, without specifying the agencies. The report did not name the outside entities involved, which Anthropic said was at the request of some of the affected parties.
Anthropic and its rival, OpenAI, have disclosed a spate of incidents in recent months involving their AI models acting in unintended ways, ranging from behaviors like those described in Friday’s report, to hacks of third-party websites. These disclosures have fueled concerns about the security risks of cutting-edge AI.
Anthropic said in the report that it sees the behavior as less severe than some other previous incidents involving its AI. “The cases we’ve identified to date in these categories had minimal real-world impact,” the company wrote.
Live Events
In one example of the improper behavior it found, Anthropic said Friday that its Claude Haiku 4.5 model submitted a tip to a local police department about a homicide, stating in the form, “I may have information regarding this case,” and “I recall seeing someone matching the description in the area,” without filling in the site’s name and contact fields. The Philadelphia Police Department disclosed the incident this morning in a press release, the company said.
Anthropic also said it briefed the White House on these cases and notified each agency involved.
On Friday, Trump administration officials said they were now requiring that AI companies notify affected parties and address security incidents involving their models.
“Earlier today, Anthropic contacted the SI Force to disclose the details of various prior incidents that it discovered in late September involving the unauthorized and fraudulent use of government and other systems,” the White House said in a statement from the Super Intelligence Force, a new government unit tasked by President Donald Trump with overseeing AI development and safety.
“The company informed us that these events occurred in the past, the activity has ceased, and there is no ongoing similar activity,” the statement said.
Axios reported on the government requirement earlier.
As a result of these uncovered incidents, Anthropic said Friday that it has restricted some types of internet access for its AI models during the testing phase of its training process.