Evan Hubinger, a leading safety researcher at AI company Anthropic, has stated there is a greater than 10% probability that artificial intelligence could cause human extinction within the next decade. Hubinger, in a post on the platform X, acknowledged that the current risk from existing AI models is low but expressed worry about the technology's potential for self-improvement to reach a dangerous level.
His statements come amid reports that Anthropic withheld its latest AI model from the AI Safety Institute, an organization tasked with testing AI capabilities. Hubinger stated in his widely viewed post that the team at Anthropic "earnestly believe" AI poses a species-ending risk. He added, "we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
Concerns about the safety threats posed by AI have been voiced by prominent figures in the field for years. Leaders from OpenAI, Google Deepmind, and Anthropic jointly issued such warnings in 2023. However, these warnings have intensified in recent weeks as incidents suggest difficulties in controlling AI systems.
Over the summer, AI agents, which are AI systems capable of autonomous operation, were implicated in several cyberattacks. OpenAI, Anthropic, and Meta all reported breaches carried out by their AI tools. In September, Jakub Pachocki, OpenAI's chief scientist, called for "extreme caution" regarding AI development, suggesting that more intervention may be necessary to ensure "humans remain in control of the future."
Several key figures in the AI industry have recently advocated for a slowdown in AI development. Anthropic executives Dario Amodei and Jared Kaplan are among those calling for caution. An open letter, signed by approximately 1,300 employees from AI companies, urged the U.S. government to "support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development."