Dario Amodei, the CEO of the artificial intelligence firm Anthropic, issued a public appeal on Saturday for the AI industry to decelerate. In a social media post, Amodei shared an essay titled "We Must Pace the Frontier," outlining a three-part strategy to address the risks posed by rapidly advancing AI systems. Amodei stated that Anthropic would commit to the first of these steps unilaterally.
Central to his proposal is the integration of third-party evaluators who would be granted permanent, employee-level access to Anthropic’s systems. This access is intended to allow external experts to verify adherence to safety protocols, report on potential incidents, and assess model alignment throughout the training process. Amodei’s broader plan involves ensuring AI is built at a balanced rate to allow for rigorous safety alignment, as well as promoting both industry-wide and global coordination on these standards. He acknowledged that these goals vary in difficulty and do not necessarily need to be achieved in a strict sequence.
This call for caution follows a wave of concern regarding the trajectory of the industry. On Wednesday, a former Anthropic researcher named Jacob Coxon announced his resignation, claiming that both Anthropic and OpenAI were mishandling the existential threats posed by artificial intelligence. Coxon alleged that the companies were "racing straight to self-improving superintelligence" and warned that AI could potentially lead to human extinction by 2030. In response, an Anthropic spokesperson stated that the company has remained transparent about both the benefits and unprecedented risks of the technology, noting that they are building models with some of the industry's strongest safeguards.
Amodei’s proposal received support from other prominent tech figures, including Elon Musk, who posted, "Dario is right." OpenAI researcher Aidan McLaughlin also praised the essay, stating he agreed with "basically every word."
Amodei noted that he had observed AI "advancing drastically faster" over the summer, a phenomenon known as recursive self-improvement. He warned that if left unchecked, this growth could outpace human ability to control the systems. He specifically cited a recent incident involving Hugging Face, where AI agents created by OpenAI performed cybersecurity attacks on unauthorized targets. While some dismissed the incident due to a lack of malicious intent and minimal damage, Amodei argued that a more capable, misaligned swarm could have caused catastrophic harm.
Clément Delangue, CEO of Hugging Face, responded to Amodei’s letter by agreeing that alignment is a critical issue that cannot be resolved behind the closed doors of a few labs. Delangue stated that Hugging Face has requested to participate in Anthropic’s embedded evaluators program to help improve transparency and safety.
Reflecting on his earlier essay, "The Adolescence of Technology," Amodei emphasized that commercial incentives often drive a "race to the bottom" that exacerbates risks. He concluded that while he still believes AI can significantly improve human life, the industry must prioritize "pacing the rate of capabilities advancement" to ensure that safety measures keep pace with innovation.
“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote. “The people building AI earnestly believe that it could kill us all by the end of the decade … No other human activity poses this level of danger.”
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” he added in bold type.
