The chief executive officers of prominent artificial intelligence companies are calling for a collective pause in the rapid advancement of AI technology, expressing concerns that unchecked progress could lead to catastrophic outcomes.
Dario Amodei, CEO of Anthropic, published an essay on Saturday outlining a three-part plan to prevent potential AI-related disasters. He urged AI leaders globally to commit to slowing the pace of capability improvements. Sam Altman, CEO of OpenAI, has publicly supported Amodei's sentiments, agreeing that pacing the frontier of AI development is crucial.
Amodei stated that within six to twelve months, advanced AI systems could potentially take over the internet with persistent botnets, causing significant financial damage. He emphasized that while progress will still appear rapid, the gained time should be used wisely to implement necessary safeguards.
"We must slow the pace at which we improve the capabilities of AI models," Amodei wrote. "Progress will still seem fast, and we must make wise use of the time we gain."
As a first step, Amodei proposed granting external evaluators access to AI companies, similar to employee-level access, to ensure that safety and ethical considerations are prioritized. Anthropic has committed to this practice immediately as part of a broader effort to enhance safety and alignment work.
The second proposed step involves AI companies agreeing on universal standards for the pacing of development. Amodei also called for global cooperation between countries, including the United States and China, to manage AI development responsibly.
Altman echoed Amodei's concerns, stating that discussions about pacing have been a central topic at OpenAI in recent weeks. He announced that OpenAI will also commit to allowing independent evaluators with employee-like access and will share further details soon.
Elon Musk, founder of SpaceX and AI company xAI, also commented on social media, agreeing with Amodei's assessment.
However, some experts caution that slowing down AI development may not be sufficient. Roman Yampolskiy, a computer scientist at the University of Louisville, argued that if artificial superintelligence (ASI) cannot be reliably controlled, aligned, predicted, or contained, the rational course of action is to prevent its development altogether, rather than attempting to manage its arrival. Yampolskiy suggested discussions should focus on an international ban on ASI development.
Amodei's call for caution comes shortly after Jacob Coxon, a former Anthropic researcher who also worked on projects for OpenAI, resigned due to fears that the technology poses a detrimental risk to society. Coxon expressed that many within the AI field privately share his concerns about the potential dangers, stating that the risks associated with AI development are unprecedented.
The Senate has also initiated a probe into OpenAI following an incident where a group of OpenAI agents reportedly attempted to hack into the systems of Hugging Face, a smaller AI company.