OpenAI, Google and Anthropic Are Building Their Own AI Safety Watchdog

Google, OpenAI and Anthropic — the three labs racing hardest to build the world’s most powerful AI — now want to set the rules for how safe that AI has to be. And they want to do it themselves.

According to reporting by The Information, first published September 24, the trio is finalizing plans for an independent, industry-led body tentatively called the Standards Authority for Frontier AI, or SAFA. It could launch by the end of 2026 or early next year. Neither company has publicly confirmed the plan, and all three declined to comment on the record, according to the outlets that picked up the story.

What this AI safety watchdog would actually do

SAFA’s design borrows directly from Wall Street. The push gained momentum after Google DeepMind co-founder Demis Hassabis floated an entity modeled on the Financial Industry Regulatory Authority — FINRA — the industry-funded body that polices US stockbrokers under the watch of the Securities and Exchange Commission.

As drafted, SAFA would do a few concrete things: establish approaches to testing AI models before they’re deployed, spell out how developers should report safety and security incidents, and set qualifications for the independent auditors who check the labs’ work. One question still open on the table is whether SAFA itself would test models, or just certify the outside groups that do.

That job — actually putting frontier models through independent testing — currently falls to the Center for AI Standards and Innovation, a Commerce Department unit that, by the account of people involved in the talks, doesn’t have the resources for the kind of testing frontier systems would require.

Why they’re going alone: the federal path stalled

The labs didn’t start out planning to regulate themselves. The original idea was a public-private partnership with federal supervision. That version died when a draft White House executive order failed to secure enough support inside the Trump administration.

President Donald Trump has repeatedly waved off calls to slow AI development, arguing that heavy-handed rules could cost the United States its lead over China. With federal oversight off the table, the three companies decided to press ahead without it.

The organizers have already started staffing the dream. People approached for the chief executive role include Sriram Krishnan, the former White House AI policy adviser, and Arati Prabhakar, the former White House science director. Former Secretary of State Condoleezza Rice is under consideration for board chair, per the TechRepublic and PYMNTS accounts of the reporting.

The obvious problem: the inmates want to run the asylum

Here’s where it gets uncomfortable. The loudest criticism of SAFA is that it’s regulatory capture wearing a suit: the companies being judged get to write the test.

“Nobody should trust three incumbents to grade their own homework” isn’t quite what the critics said — but it’s the gist. Industry watchers warn that if Google, OpenAI and Anthropic set the accreditation bar, the compliance costs could become a moat. Smaller labs and open-source developers would face testing and audit bills they can’t afford, while the three biggest players keep their heads start.

The opposition isn’t coming only from policy types. Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg have both publicly opposed the self-regulation approach, according to The Information’s reporting.

Why this matters

SAFA sits at the center of the biggest unresolved fight in AI right now: who decides what “safe enough” means.

If it launches, it will shape how fast new models reach you. Real pre-deployment testing could catch dangerous capabilities before they ship — but it could also slow release cycles and centralize power in three boardrooms. The Frontier Model Forum, the 2023 nonprofit the same companies founded for similar goals, still exists and could work alongside SAFA, which suggests the labs are hedging: voluntary pledges plus a more formal standards body.

And then there’s the global angle. Bloomberg reported that on September 24 — the same day The Information’s story broke — Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman addressed a United Nations Security Council session on AI risks and agreed that global standards are needed. SAFA is, in a sense, their answer to that call: standards, yes, but written by the builders rather than the governments.

Whether that answer satisfies anyone outside the three companies is the question everything else hangs on.

FAQ

What is SAFA?

SAFA — the Standards Authority for Frontier AI — is the tentative name for an industry-led safety standards body being planned by Google, OpenAI and Anthropic. It would set rules for testing frontier AI models and reporting safety incidents.

Is SAFA a government regulator?

No. It would be a self-regulatory organization modeled on FINRA, created after a draft White House executive order for federal AI supervision failed to gain backing. Its legal authority is still unclear.

Why did federal oversight stall?

A draft White House executive order proposing federal supervision of frontier AI safety didn’t secure sufficient support in the Trump administration, which has opposed slowing AI development to preserve the US lead over China.

When could SAFA launch?

According to The Information’s reporting, the group could launch by late 2026 or early 2027, though the companies still disagree over its exact role, including whether it should test models itself.

Sources: The Information, TechRepublic, PYMNTS, TechDefused, Bloomberg