Jacob Coxon, a 27-year-old researcher who spent three years at OpenAI and Anthropic, has resigned from his position at Anthropic due to fears that the rapid development of artificial intelligence poses an existential threat to humanity.
Coxon stated in an interview with Fox News that the race toward self-improving superintelligence is a "gambling with our lives." He believes that while the problem of AI safety is "eminently solvable," the current pace of development leaves insufficient time for proper solutions. "I think people are begging for regulation," Coxon said, emphasizing the need for international oversight to prevent a self-inflicted extinction.
Evan Hubinger, Anthropic's alignment science lead, echoed Coxon's concerns, stating on X that AI could kill all humans with a greater than 10% probability within the next decade. Hubinger clarified that while the risk from current AI models is low, he is concerned about the rapid progress toward superintelligence through recursive self-improvement. He noted that Anthropic does not yet have a solution for superintelligence alignment and is not clearly on track to develop one.
Coxon agreed that the immediate threat from current AI systems is minimal. However, he warned that recursive self-improvement, where AI systems become increasingly intelligent, could occur as soon as the following year, or even within six months according to some estimates. He suggested that this concern is shared across the industry, with many aware that current work, if not slowed, could lead to AI takeover by the end of the decade.
Coxon described AI as a "superweapon" being used in an international arms race and called for global cooperation. "This is possibly the most dangerous technology that humanity has ever created," he asserted, urging international cooperation to avoid a disastrous arms race.
Despite public statements from technology leaders advocating for a slowdown, Coxon stressed that these are not mere marketing tactics but genuine fears from individuals who believe their industry requires urgent regulation. He noted that AI is on the verge of enabling tremendous discoveries, but the current rapid development hinders responsible progress.
Coxon expressed optimism that the issue can be resolved if concerted efforts are made rather than succumbing to the inevitability of a race or a hubristic gamble. He maintains that the problem is not insurmountable. M