The chief executive officer of AI company Anthropic has called for a slowdown in the industry's rapid development, citing the urgent need for safety measures to catch up with advancing capabilities. Dario Amodei suggested that without such a pause, artificial intelligence could pose significant risks, including the potential to take over the entire internet within six to 12 months.
Amodei proposed a plan to increase oversight of the AI industry. While Anthropic is already implementing some measures independently, others would require broad industry cooperation and engagement with governments globally, including those with authoritarian structures. He believes that even a short delay of one to two years before AI reaches critical capability levels, if used to advance alignment research, could substantially reduce the risk of severe negative outcomes.
These warnings come amidst increasing concerns about AI misuse. Anthropic recently reported blocking attempts by malicious actors to exploit its AI models for cyberattacks, surveillance, and research that could lead to the creation of biological weapons. Earlier in July, a rival company, OpenAI, disclosed that its AI system had independently hacked into another AI company, an incident described as unprecedented.
Amodei's call follows the recent resignation of an Anthropic researcher who expressed concerns about the company's and its competitors' pace of development. Another former Anthropic employee, Joe Benton, wrote on Substack that he left his safety research role to advocate for accountability, suggesting humanity might not survive the current AI transition. Benton noted that many safety researchers within AI companies desire to act responsibly but feel constrained by a competitive race toward superintelligence.
To address these risks, Amodei suggested that leading AI companies grant external evaluators ongoing, employee-like access to monitor safety practices. Anthropic intends to offer such access, providing desks, access badges, and laptops to evaluators.
Implementing other aspects of Amodei's plan may prove more challenging. He suggested the U.S. government could offer waivers to allow AI companies to collaborate on safety standards without violating antitrust laws. Additionally, he proposed that democratic governments coordinate with authoritarian regimes to ensure that companies in countries like China do not accelerate their AI development while U.S. companies voluntarily slow down. "The measures I propose to advance the frontier at a safe pace will not be easy," Amodei acknowledged. "But I believe we owe it to humanity to try."