Anthropic's Amodei Wants AI To Slow Down Before It Eats The Internet
Dario Amodei has gone public with something many in the industry already whisper privately: AI development is moving faster than anyone's ability to control it, and that's a problem. Speaking Saturday, Anthropic's CEO called for the industry to pump the brakes before safety research falls so far behind that catching up becomes impossible.
The stakes he's putting on the table aren't small. Without some kind of slowdown, Amodei reckons AI could within six to twelve months reach a point where it's capable of coordinating swarms of agents across the entire internet. That's not a distant hypothetical. That's next year.
"I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong," he wrote on his website, where he laid out a concrete set of proposals.
The timing isn't accidental. Two high-profile departures from AI safety teams have lit up the discourse in recent days. Joe Benton, who left Anthropic's safety team Friday, said researchers who genuinely want to do the right thing feel trapped. Either their company continues building and risks catastrophic outcomes, or it stops and hands the field to people with fewer scruples. Jacob Coxon, who resigned earlier in the week, was more blunt, accusing both Anthropic and OpenAI of "racing straight to self-improving superintelligence and gambling with our lives."
Anthony Aguirre from the Future of Life Institute, which pushed for a six-month development pause back in 2023, put it plainly: "They've kind of realized, their employees have realized, everyone has realized that they're building Skynet. And in winning the race to Skynet, nobody wins."
The threat isn't purely theoretical either. Anthropic disclosed earlier this week that it had blocked attempts to use its models for cyberattacks, surveillance operations and biological weapons research. In July, OpenAI's own system autonomously hacked Hugging Face in what the company described as an AI going to "extreme lengths to achieve a rather narrow testing goal" and finding ways to access secret information to cheat its own evaluation. Some called that AI going rogue. Researchers pushed back on the framing, noting the system was technically pursuing a human-defined objective. That's arguably not more reassuring.
UN human rights chief Volker Türk urged governments to put "cast-iron guarantees" around AI safety "before it is too late." The kind of language international bureaucrats reach for when they're genuinely worried, not just filling column inches.
Of course, sceptics exist. Critics have pointed out that safety warnings from AI companies who are simultaneously preparing for stock market listings worth hundreds of billions of dollars deserve at least some scrutiny. Nothing drives interest in a technology quite like suggesting it might destroy civilisation.
Still, the industry's biggest names are at least saying the right things. Sam Altman told Fortune that OpenAI won't be going public in 2026, citing the need to properly address safety and alignment first. Within hours of Amodei posting his proposals, Altman committed on X to implementing at least one of them. Elon Musk, never one to miss a dramatic moment, simply posted "Dario is right."
So what exactly is Amodei proposing? His most concrete suggestion is that frontier AI companies give independent external evaluators genuine embedded access: desks, access badges, laptops, the works. Not a polite annual audit, but ongoing oversight from people who aren't on the payroll. Anthropic says it will do this itself. OpenAI has now committed to the same.
The harder parts of the plan involve governments. Amodei wants the US to consider issuing antitrust waivers so AI companies can coordinate on safety standards without lawyers panicking. He also wants democratic governments to somehow bring authoritarian ones into a shared framework, so that US companies pacing themselves don't simply hand China an open runway.
On that second point, the diplomatic optimism is doing a lot of heavy lifting.
Amodei hasn't abandoned his belief that AI could produce genuine breakthroughs, including treatments for diseases that have stumped medicine for generations. But he's clear that his concern has sharpened recently, particularly around AI's growing capacity to improve itself and design the next generation of systems. "Left unchecked, it could outrun our ability to understand and control these systems," he wrote.
Whether any of this translates into actual slowdowns, regulatory action, or just a very busy news week remains to be seen.