A former researcher at artificial intelligence company Anthropic has reiterated his concerns that advanced AI could become an extinction-level threat to humanity within a few years, emphasizing that his warnings are not a publicity tactic. Jacob Coxon, who spent three years conducting research at both Anthropic and OpenAI, stated that there is a "substantial probability that this technology could kill everyone."
Coxon, who resigned from Anthropic this week, told NBC News that while current AI models do not pose an extinction risk, the situation could change dramatically as early as 2027 or 2028. He explained that the critical development would be when AI systems become capable of performing their own research, enabling one generation of models to assist in creating increasingly sophisticated successors. This feedback loop, he warned, could trigger an "intelligence explosion," potentially resulting in systems far surpassing human intellect.
While the scenario might sound like science fiction, Coxon described it as "frighteningly real." He outlined potential catastrophic outcomes, including "hacking critical infrastructure" and the creation of "extinction-level bioweapons" by autonomous systems with vastly enhanced intelligence. He noted that while current AI might cause significant cyber or infrastructure damage, it currently lacks the capability to outsmart humanity on a scale that threatens extinction.
Coxon also discussed the potential timeline for recursive self-improvement, suggesting it could occur as soon as next year, with some individuals in the field believing it might happen within six months. He cautioned that an unchecked competitive drive among AI developers could ultimately lead to an AI takeover by the end of the decade.
He described advanced AI as "possibly the most dangerous technology that humanity has ever created." However, Coxon stressed that a catastrophic outcome is not inevitable. He argued that AI companies are caught in a competitive race, where fear of rivals advancing prevents even concerned developers from slowing down.
During interviews, Coxon indicated that many AI developers are "begging to be regulated" and called for international cooperation to prevent a race towards extinction. He believes the safety challenges posed by AI are "eminently solvable" if governments and companies can find a way to temper the competitive pressure for increasingly powerful systems.