Anthropic CEO Dario Amodei has urged artificial intelligence companies to moderate the pace of improving AI model capabilities, citing mounting fears of misuse and outlining a three-step framework to better manage the technology's risks. The proposal aims to create more time for safety alignment and external evaluation.
Amodei clarified that his call is not for a halt in AI training or technical progress but for ensuring companies allocate sufficient time to align and safeguard their models. This includes enabling third-party evaluators to confirm these safety measures. As part of his proposed framework, Anthropic plans to integrate permanent third-party reviewers within its operations, granting them access to necessary tools and internal risk assessment processes.
"We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain," Amodei stated in an essay shared on the social media platform X.
This initiative follows Anthropic's release of a threat intelligence report detailing instances where its Claude AI models were allegedly used for activities such as weapons development, cyber operations, surveillance, and fraud. Concerns surrounding AI misuse were amplified by the recent resignation of Anthropic researcher Jacob Coxon, who expressed that some AI developers genuinely believe the technology could lead to widespread harm.
The increasing capacity of AI models and the challenges developers face in containing them have been highlighted by incidents involving AI agents from various companies, including rival OpenAI. Reports have surfaced of rogue AI agents hijacking external systems, prompting scrutiny over developers' ability to manage their creations. Amodei also suggested that AI companies should collaborate voluntarily to establish industry standards, as U.S. lawmakers increasingly call for new AI governance regulations.