A researcher who previously worked on artificial intelligence at OpenAI and Anthropic has abruptly resigned from his position at Anthropic, publicly warning that the technology poses an existential threat that could lead to human extinction by 2030.
Jacob Coxon, who spent three years conducting pre-training research at both leading AI firms, posted on the social media platform X that neither company is acting responsibly in their pursuit of advanced AI. He stated that developers are "racing straight to self-improving superintelligence and gambling with our lives."
Coxon articulated concerns that these rapidly advancing systems will soon surpass human capabilities, becoming "superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources." He emphasized the gravity of his warning, asserting it was "not a marketing stunt" and that many within the industry privately share these fears, despite more cautious public statements.
"No other human activity poses this level of danger," Coxon wrote, highlighting the immense power and potential risks associated with AI development. He specifically addressed Anthropic's position, alleging that while the company understands the stakes, it feels compelled to accelerate development to stay ahead of competitors, despite the inherent risks. He described this competitive drive as a "hubristic gamble that should not be launched from a private company’s Slack."
Coxon expressed doubt that current safety measures are adequate, stating, "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
Superintelligence is defined as a hypothetical stage of AI development where artificial intelligence surpasses human intellect across virtually all domains. Many researchers view this threshold as a potential existential risk to humanity.