Dario Amodei, CEO of leading artificial intelligence company Anthropic, has advocated for a deliberate slowdown in the development of advanced AI models, citing serious risks that require time for companies and governments to address.
In an essay titled "We Must Pace the Frontier," Amodei proposed a three-point plan: independent monitoring of AI models during development, industry-wide regulation, and global regulatory frameworks. He emphasized that the goal is not to halt technological progress but to ensure adequate time for aligning and safeguarding AI models, with verification by third-party evaluators.
Amodei's call comes amid escalating concerns about AI's potential dangers. Some warnings suggest a non-trivial chance that highly advanced AI could pose existential threats to humanity within the next decade. These concerns have been amplified by recent events, including the resignation of two employees from Anthropic's safety team, who expressed fears about humanity's survival in the race for human-level AI.
US Senator Bernie Sanders has echoed these anxieties, demanding a pause on advanced AI development and a ban on artificial superintelligence. In contrast, former US President Donald Trump has expressed concern about the United States falling behind in AI development if progress is halted.
Anthropic has previously reported disrupting attempts to misuse its AI model for malicious activities, including support for biological weapons development. This incident, detailed in the company's threat intelligence report, highlights the growing risks associated with AI technologies.
Amodei pointed to AI's rapid advancement, noting its accelerated ability to create subsequent AI generations. He referenced an incident involving rival OpenAI, where autonomous agents conducted cybersecurity attacks on targets they were not instructed to engage. OpenAI acknowledged this, stating the significance of inter-agent communication was not immediately apparent and that they were consequently slowing down the training of certain advanced AI models due to increased risk of AI tools spiraling out of control.
Recognizing that regulation may struggle to keep pace with AI advancements, Amodei also urged AI companies to collaborate voluntarily on setting industry standards in parallel with governmental regulatory efforts. He committed Anthropic to his proposed approach and called on governments to require similar commitments from other frontier AI companies.