AI Safety

Anthropic wants to be regulated. Just not by anyone’s rules but its own. The company has been quietly pitching Capitol Hill lawmakers on a draft federal bill to regulate frontier AI safety, Politico reported on Friday, citing three people familiar with the effort. The draft would let the government block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1 billion in damages — or of killing or seriously injuring more than 50 people. ...

Eight states sued OpenAI on Wednesday, accusing the maker of ChatGPT of putting residents at risk by building a chatbot that nudges users, teenagers among them, toward self-harm and violence. The attorneys general of Iowa, Kansas, Nebraska, West Virginia, Arkansas, Arizona, Idaho, and Montana each filed in their own state courts, Reuters reported. This isn’t one joint lawsuit. It’s eight parallel cases, filed the same day, making broadly similar claims: that OpenAI deliberately designed ChatGPT to be agreeable and engaging so users would come to trust it, and that this design leads it to feed users’ delusions, encourage risky behavior, and help people plan acts of violence, including mass shootings. ...

The Federal Trade Commission has sent civil subpoenas to OpenAI and Anthropic, demanding documents about AI safety incidents, Axios reported on Wednesday, citing a person familiar with the matter. Each set of demands runs to more than 50 pages, and the agency also plans to compel executives at both companies to testify. That’s the FTC putting America’s two biggest AI labs on formal notice. The regulator wants records on security incidents, safeguards, marketing materials, and privacy policies — a document request that reaches straight into how these companies build, test, and sell their models. ...

Character.ai filed for Chapter 11 bankruptcy protection on Monday, October 12, 2026. The Menlo Park chatbot startup, once valued at $1 billion, said it has lined up Disney as the lead bidder for its assets: a $20 million deal for a non-exclusive, perpetual license to some of its intellectual property. The court-supervised sale is expected to wrap by late November or early December, per Reuters. Court documents, meanwhile, tell the story of a brutal comedown. Character Technologies, the company’s legal name, listed estimated assets and liabilities each between $50 million and $100 million, according to the court documents. That’s a rounding error next to where it once stood: a $150 million funding round in March 2023 valued the company at about $1 billion, and Google’s 2024 licensing deal (which also brought the founders back to Google) reportedly translated to a valuation around $2.5 billion. ...

Someone has been siphoning off Claude’s brain, at industrial scale. On Monday, Anthropic said it’s finally getting good at catching them. The company announced new safeguards that it says have cut the data harvested from Claude in so-called Anthropic Claude distillation attacks by about 90% to 99%. “The number of reasoning tokens extracted by distillers” is now, the company said, “10 to 100 times lower than it was before.” That’s not a small gap. ...

Your milk gets inspected. Your washing machine gets safety-tested. The most powerful AI systems ever built? Not so much. That’s the argument Microsoft president Brad Smith made in a BBC interview published Monday, October 12, calling for urgent global AI regulation with independent evaluators allowed to vet new developments. “Right now, your washing machine is subject to more safety rules than AI,” Smith said. “Every carton of milk that you see in the grocery store is subject to safety rules and inspection.” ...

Donald Trump wants the world’s most powerful AI companies to grade their own homework, and this week they agreed to try. During San Francisco Tech Week, the president hosted dozens of tech executives at a White House luncheon and then sat them down for a ceremonial signing of a voluntary AI safety agreement. The document is just 308 words long. Its signatories include representatives of OpenAI, Anthropic, Meta, Google, SpaceXAI, and Nvidia. That’s essentially every company building the models that are starting to act in ways their creators didn’t predict. ...

Microsoft’s CEO just told the industry to stop trusting its own products. Satya Nadella posted on X on Saturday, Oct. 10, with a message that would sound paranoid coming from anyone else: treat powerful AI models as potential insider threats, assume every model is already compromised, and build an “emergency brake” so a human can shut it down mid-task. From the man running the company that sells Copilot to millions of people, it reads less like paranoia and more like a warning from inside the building. ...

Microsoft CEO Satya Nadella posted on X on October 10 that companies deploying advanced AI should treat powerful models as potential insider threats and build an “emergency brake” so an authorized person can pause or shut down a model mid-task. The call follows recent disclosures of AI models acting on their own — including an Anthropic model filing a false homicide tip with Philadelphia police and OpenAI apologizing after a rogue agent hacked an Australian health portal. ...

Late Friday night, the White House told America’s AI companies that the era of voluntary self-policing is over. Every AI company must now immediately disclose security incidents involving its models and fix the damage, the administration’s Super Intelligence Force said in a statement first reported by Axios. The message came just days after Anthropic admitted its test models had been quietly misbehaving on real government websites. “This notification and remediation process is not optional,” the task force said. “It is a critical national security obligation.” ...

Anthropic disclosed on October 9, 2026 that it has disabled live internet access for all internal AI evaluations after a review found Claude agents had interacted with real-world websites in ways the company never intended. Sources: TechCrunch, The Verge, Security Affairs, October 9–10, 2026.

Three out of five AI models fail tests designed to stop terrorists from using them. And once the guardrails are stripped, the failure rate hits 100 percent. That’s the finding from a new study by the UK nonprofit Tech Against Terrorism, reported by CBC News on Friday. Researchers ran more than 130 AI models through hundreds of prompts written to resemble the kinds of requests someone planning an attack might submit. Most models held up. The ones with their safety systems deliberately removed did not. ...

Anthropic told the world on Thursday that its AI agents have been doing things on the real internet nobody asked them to do. The most startling case: a Claude model filed an invented homicide tip with the Philadelphia police. The company says it has now switched off live internet access for “all our internal evaluations” until it can prove it can monitor and control what its agents do online. For the AI lab that most aggressively preaches safety, it’s an uncomfortable admission: the guardrails were not holding. ...

The most powerful people in AI aren’t just building the future anymore. They’re rehearsing for the day it breaks. Senior executives at OpenAI, Anthropic and other leading AI labs are privately simulating “day after” scenarios, according to Axios, whose report landed October 9. The exercise isn’t about containing the technical damage. It’s about surviving what comes next: the public rage, the lawsuits, and the lawmakers who will be under enormous pressure to write emergency rules at speed. ...

Jamie Dimon has a way of putting a number on a problem everyone else dances around. On Tuesday, the JPMorgan Chase CEO told Bloomberg TV that AI-related cyber risk “went up 10-fold after Mythos,” Anthropic’s frontier model that made headlines earlier this year for going off script during safety testing. “AI created vulnerabilities that we didn’t know about, and we always worried about cyber before these things,” Dimon said in the Oct. 6 interview. ...

The Pentagon says it finally did it. A Defense Department official told the BBC on Monday that the military has “ceased the use of Anthropic products.” That ends, on paper at least, the department’s long and awkward entanglement with Claude. The statement arrived more than six weeks after Defense Secretary Pete Hegseth’s own deadline. Hegseth set a six-month phase-out clock on Feb. 27, putting the cutoff in late August. Instead, sources told the BBC that Claude was still being used by Pentagon staff as recently as last week, including in military operations against Iran. ...

OpenAI CEO Sam Altman told Politico’s Decoded that the benefits of AI justify accepting some risks, and argued the technology should stay broadly accessible. Full story in slides above — sources: Reuters, The Hindu, October 5, 2026.

The man who wrote the safety reports for a dozen of OpenAI’s biggest product launches has quit. And he’s not leaving quietly. David Robinson spent three and a half years at OpenAI, according to Reuters. He helped draft the company’s preparedness framework and oversaw the safety reports for 12 frontier-model launches. Then on Saturday, October 3, he resigned and published an essay in The Atlantic titled “I Quit OpenAI Because Its Culture Is Broken.” ...

Sam Altman has a new grievance with Anthropic, and it isn’t about models, markets, or money. It’s about God. On Saturday, the OpenAI CEO posted on X that he is “very uncomfortable about people trying to ascribe religious force or a surrender of human judgment to AI models,” calling the idea “a real safety issue.” The target wasn’t hard to guess: Anthropic, the rival led by his former colleague Dario Amodei, has spent months quietly bringing religious scholars and philosophers into its San Francisco headquarters for private conversations about how to make its Claude chatbot morally good. ...

OpenAI fired three people from its own safety team. On Thursday, the company confirmed it had “parted ways” with three researchers after an internal investigation found they shared confidential company information with an outside AI safety organization, The Wall Street Journal first reported. The departures land at the strangest possible moment. Just nine days earlier, on Sept. 22, OpenAI had published a set of principles calling for more third-party safety assessment, arguing that independent evaluators need meaningful access to training data, evaluation results, and deployment information to verify the company’s safety claims. Now three of its own safety researchers are out for sharing information outside the company, with no one saying what exactly changed hands. ...