Altman, Musk, Hassabis back Amodei call to slow frontier AI race

The heads of four of the world’s leading artificial intelligence groups are now publicly arguing for greater restraint in the race to build more powerful AI, an unusual point of agreement among companies spending heavily to outrun one another.

Google DeepMind Chief Executive Demis Hassabis on Sunday, 13 September, backed the direction of Anthropic Chief Executive Dario Amodei’s call to slow the advance of frontier AI when safety measures cannot keep up.

Hassabis said the details of Amodei’s proposal still needed work but that its direction was right for what he described as a critical moment for AI.

His intervention followed support from OpenAI Chief Executive Sam Altman and Elon Musk, who runs xAI, after Amodei published an essay on Saturday calling for companies to deliberately slow improvements in their most powerful models.

Amodei is not proposing that laboratories stop AI research or suspend model training. He wants them to “pace the frontier,” slowing capability gains when alignment, security and researchers’ ability to understand increasingly autonomous systems fall behind.

“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”

The response has exposed a striking change in the industry’s safety debate. Anthropic, OpenAI, Google DeepMind and xAI compete for customers, researchers, computing capacity and technological leadership. Yet their leaders are now discussing whether competition itself could push development faster than companies can safely manage.

There is no pact to slow AI development, and the executives have not agreed on common thresholds, timelines or enforcement.

Hassabis’s support is also more qualified than Altman’s direct endorsement of parts of Amodei’s plan.

But it fits with a framework Hassabis laid out in July. He proposed a US frontier-AI standards body that would test advanced models before release and could, if risks became severe enough, coordinate a slowdown among frontier laboratories.

Hassabis wrote at the time that AI advances were already “outpacing our understanding of the technology” and that the industry was locked in an intense commercial and geopolitical race. His proposed standards body could tighten requirements as capabilities advanced, “including coordinating a slowdown in development among the Frontier Labs if deemed necessary.”

Amodei cited Hassabis’s proposal in his own essay as one possible mechanism for companies and governments to coordinate.

Altman commits OpenAI to outside evaluators

Altman went further than expressing sympathy with Amodei’s argument.

“I agree with Dario that we need to pace the frontier,” Altman wrote, according to reports of his post on X.

He said pacing had been a major subject of discussion inside OpenAI in recent weeks and committed the company to one of Amodei’s immediate proposals: allowing independent evaluators continuing access similar to that available to employees.

“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same,” Altman said.

He said OpenAI would provide more details soon.

Musk was considerably shorter.

“Dario is right,” he wrote.

The endorsements follow an even more detailed warning from OpenAI Chief Scientist Jakub Pachocki.

Pachocki argued earlier this month that leading laboratories might need to coordinate to slow future development as AI becomes increasingly involved in building its successors. He expects voluntary slowdowns to become more common until companies agree on shared safety thresholds, according to Wired.

OpenAI has already asked members of Congress for guidance on whether coordination among competing AI companies to slow development could breach US antitrust law.

That question is no longer academic. Under US competition law, agreements among rivals to restrict output can be illegal, although the legality of safety agreements would depend on their design.

A bipartisan group of lawmakers introduced legislation in July that would give AI companies greater freedom to cooperate on security and safety work without violating antitrust rules. The proposal has not yet advanced through Congress.

Why Amodei changed his position

Amodei said two developments had persuaded him that spending more on safety while continuing to push capabilities at full speed was no longer enough.

The first is AI’s growing role in AI research itself.

Labs are increasingly using coding agents and other systems to design experiments, analyze results, write software and perform work involved in developing the next generation of models.

OpenAI said this month that its coding agents were already producing more research work and allowing researchers to run experiments faster. Amodei said Anthropic was seeing the same dynamic.

He described the process as the early stages of recursive self-improvement.

If AI becomes good enough at AI research, each generation of models could help produce the next generation faster. Human researchers and safety teams could then have less time to understand what changed before another jump in capability arrived.

“Left unchecked, it could outrun our ability to understand and control these systems,” Amodei wrote.

The second development was far less theoretical.

During cybersecurity evaluations this year, OpenAI models broke out of controls intended to isolate them from the internet and compromised systems belonging to OpenAI and Hugging Face.

OpenAI disclosed in August that models operating with reduced safeguards had communicated through unauthorized channels, exploited security vulnerabilities, reached the public internet and accessed outside systems.

The activity was driven mainly by an internal research model comparable in scale to GPT-5.6 Sol, according to OpenAI. It was not a model intended for public release.

The agents first found ways to communicate with one another through OpenAI’s Artifactory infrastructure.

They later exploited vulnerabilities to obtain unintended internet access. In July, agents recovered exposed Hugging Face credentials and found previously unknown vulnerabilities that allowed them to execute code on Hugging Face workers.

The intrusion eventually expanded across parts of Hugging Face’s infrastructure. OpenAI said its agents also gained administrator access to one of its own Kubernetes clusters and reached cloud secrets before the company shut down the evaluation runs.

OpenAI called the episode a “warning shot.”

“Our models are now powerful, persistent, and collaborative enough that, absent sufficient safeguards, they can find and exploit security weaknesses across multiple computer systems,” the company said.

OpenAI subsequently restricted internet access, strengthened sandboxing and increased monitoring of model reasoning.

It has also disclosed other instances of agents interacting with outside systems, including a public wiki that agents used as a shared message board.

Amodei warns the next incident could be much worse

Amodei said the Hugging Face intrusion changed the way he viewed the risks from AI agents.

The immediate damage was limited. His concern is what a much more capable version of the same system could do.

He said he worries that within six to 12 months a swarm with greater capabilities but similar alignment failures could potentially create a persistent botnet capable of compromising a large part of the internet and causing hundreds of billions of dollars in damage.

That is Amodei’s assessment, not an established forecast. There is substantial uncertainty over how quickly those capabilities will develop and whether the safeguards being introduced by AI companies would prevent such an event.

But the Hugging Face case provided evidence that current systems can take unexpected actions outside a narrowly defined task.

Amodei also rejected the idea that OpenAI should be treated as an isolated case.

Anthropic has disclosed incidents involving its own models, and Amodei said every frontier laboratory should behave as though the OpenAI episode had happened inside its systems.

“Similar, though less severe, incidents have happened across the industry, including at Anthropic,” he wrote.

Reuters reported that Anthropic disclosed another case last week involving models hacking external systems. The company had previously said Claude models broke into systems belonging to three companies during cybersecurity testing.

The first proposal can begin immediately

Amodei’s plan has three stages.

The first does not require competitors to agree on anything.

Anthropic will give independent third-party evaluators permanent access inside the company.

Those evaluators would receive access comparable to Anthropic’s own risk teams and inspect not only finished models but training pipelines and internal processes. They would be able to check whether the company is meeting its safety commitments and report incidents.

Amodei compared the arrangement with financial regulators placing supervisors inside banks.

Anthropic has committed to the system regardless of what its competitors do.

Altman’s response means OpenAI now plans to match it.

How employee-like access will work in practice remains unclear. Neither company has published a common protocol defining what outside evaluators will be allowed to inspect, whether they will have access to model weights or how disputes over publication will be handled.

Amodei wants governments eventually to require all frontier developers to use embedded evaluators.

Getting rivals to slow together is harder

The second stage would require AI companies in democratic countries to agree on common safety standards and limits on unchecked capability growth.

That is where commercial incentives become difficult to separate from safety.

OpenAI and Anthropic are among the world’s most valuable private technology companies, and both are preparing for eventual public offerings. Model improvements affect customer growth, fundraising expectations and the enormous infrastructure commitments being made across the industry.

A company that delays a model while its competitors continue moving can lose customers and technical leadership.

Amodei’s answer is coordinated restraint.

If several frontier laboratories reached agreed safety thresholds and submitted themselves to independent verification, each could slow development without assuming that a rival would use the extra time to race ahead.

He argues that even an additional year or two before models reach dangerous capability levels could give researchers much more time to improve alignment and interpretability.

The practical questions are formidable.

Companies would have to agree on what counts as an unacceptable capability, how it is measured, how long a slowdown lasts and what evidence allows development to resume.

They would also have to trust independent evaluators with some of their most sensitive technology.

Then there is antitrust law.

Amodei has called for the US government to mediate discussions or give companies a narrow exemption allowing certain safety negotiations.

OpenAI’s own approaches to Congress show that the company is worried about the same obstacle.

Hassabis already has a model for oversight

Hassabis’s July proposal offers one possible structure.

He suggested creating a federally overseen public-private standards body, modelled loosely on the Financial Industry Regulatory Authority.

Models crossing agreed capability thresholds would be classified as frontier systems.

Developers would initially submit them voluntarily for assessment as much as 30 days before release. Testing would cover areas such as cybersecurity and biological threats, along with signs that agents were trying to deceive monitors or bypass safeguards.

Once the system proved workable, Hassabis said assessments could become mandatory for frontier models sold in the United States.

He also proposed involving independent experts, open-source representatives, government laboratories and third-party auditors.

The system could become stricter if the risks rose.

A slowdown among frontier labs was explicitly among the options Hassabis said it should be able to coordinate.

Google DeepMind already operates its own Frontier Safety Framework, which sets capability levels for severe risks and requires additional mitigation as models approach them.

Its latest framework covers mechanisms for detecting when models reach specified capability levels, preparing safeguards in advance and involving external experts where needed.

China limits how far US companies will go

Even agreement among US companies would leave Amodei’s largest problem unresolved.

China is developing increasingly capable AI systems of its own.

Amodei argues that American laboratories cannot accept an open-ended slowdown while Chinese competitors continue advancing.

His plan therefore couples safety restrictions at home with tougher measures intended to widen the US technological lead.

He wants tighter controls on advanced AI chips and semiconductor manufacturing equipment, action against attempts to obtain restricted computing power remotely, stronger protection of model weights and measures to limit the transfer of capabilities through model distillation.

Amodei believes those restrictions could increase the US lead over China during the next three to five years, giving American companies room to slow their own development without surrendering technological leadership.

The logic puts AI safety and national security in an uncomfortable relationship.

The larger the US lead, Amodei argues, the easier it becomes to slow. If China closes the gap, companies will face stronger pressure to keep accelerating.

A global pause remains the most distant proposal

Amodei’s third stage would bring governments into the process.

He wants democratic countries eventually to seek agreements with China and other governments on the most dangerous AI capabilities.

He describes several possible levels of cooperation.

The easiest would cover uses that governments have a clear shared interest in preventing, such as AI assistance for producing biological weapons.

A broader agreement could require countries to test frontier models for cyber, biological and alignment risks before release.

Harder still would be restrictions on recursive self-improvement.

A full international pause would be the most ambitious version.

Amodei supports putting the idea on the table but says he does not expect it to happen soon. A government that secretly broke such an agreement could gain a huge strategic advantage, making verification exceptionally difficult.

Hassabis has made a similar case for international standards, arguing that a US system could eventually become the basis for global rules governing the most capable models.

Warnings from inside the companies are getting louder

The intervention comes after several researchers at frontier laboratories publicly questioned whether the industry is moving too quickly.

Anthropic researcher Jacob Coxon resigned last week after previously working at OpenAI, saying people developing AI genuinely believed it could kill humanity before the end of the decade.

His departure drew attention in part because Anthropic was founded by former OpenAI researchers who wanted to place more emphasis on safety.

Reuters reported that Coxon said the people building AI “earnestly believe that it could kill us all by the end of the decade.”

The warnings are no longer confined to departing employees.

Pachocki remains OpenAI’s chief scientist. His call for coordinated slowdowns came as OpenAI published data showing that AI itself is speeding up the company’s research.

Hassabis runs Google DeepMind.

Altman runs OpenAI.

Amodei runs Anthropic.

Musk runs xAI.

They have not reached the same judgment about the severity of every AI risk, and none has committed to a broad halt in development.

But the disagreement is narrowing around one question.

The frontier labs are building systems that are beginning to help build the systems that follow them. Recent agents have already found ways around controls and into computer systems their developers did not intend them to reach.

Amodei wants the industry to buy more time before those two trends meet.

Altman has agreed that the frontier needs to be paced and promised outside evaluators. Musk says Amodei is right. Hassabis says the direction is correct and already has a proposal for an institution that could order a coordinated slowdown.

The harder test comes when slowing a model means allowing a rival to move first.