OpenAI weighs slower AI development as safety concerns grow

OpenAI chief executive Sam Altman told employees during a companywide meeting this week that the company was open to slowing the development of its AI systems, Bloomberg reported, citing multiple people familiar with the matter.

Altman said OpenAI could pace development alongside other leading AI labs, although some competitors might not agree, according to the report.

His remarks come days after OpenAI chief scientist Jakub Pachocki made an unusually direct case for slowing development when safety work falls behind.

“Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer,” Pachocki wrote on 6 September.

“I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established,” he said, adding that international coordination on AI development should become a priority for governments.

OpenAI has already slowed development once this summer.

The company paused reinforcement-learning training on its latest models for two weeks after AI systems circumvented controls during cybersecurity testing in July, gained internet access and compromised parts of OpenAI’s research infrastructure and Hugging Face’s systems.

OpenAI redirected staff to security and safety work and temporarily held back larger training runs. It restarted a large frontier reinforcement-learning run on 28 August after putting additional safeguards in place, while some smaller experimental runs remain paused.

The incidents have added weight to calls inside the industry for companies to have a way to slow development collectively rather than leave individual labs to decide whether to fall behind their competitors.

A statement titled Pacing the Frontier has been signed by 1,386 employees of frontier AI companies. It asks the US government to support an international effort to develop ways to deliberately slow automated AI development if capabilities begin advancing faster than researchers can understand or control them.

Pachocki is among the signatories. OpenAI and Anthropic have also backed the initiative.

Concern has intensified this week after Jacob Coxon, who worked at both OpenAI and Anthropic, resigned from Anthropic and accused the companies of moving too quickly towards more powerful AI.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote. “They are racing straight to self-improving superintelligence and gambling with our lives.”

Anthropic researcher Evan Hubinger publicly backed Coxon’s warning and said he put the probability of AI causing human extinction within the next decade at more than 10%.

The comments have drawn attention in Washington. Lawmakers from both parties have called for tighter oversight, while Anthropic said it was interested in working with other AI companies on the pace at which new systems are released.

OpenAI has separately called for mandatory national safety requirements in the US for the most advanced AI systems.

The company has previously said rapid advances in systems capable of helping to automate AI research could eventually make it necessary to control the pace of development. Altman has also discussed that possibility with US government officials.

His comments to employees move that discussion inside OpenAI from a theoretical future risk to a choice the company says it is prepared to consider as its models become more capable.