Multiple employees at artificial intelligence company Anthropic have raised alarms about the safety risks associated with AI development, echoing concerns voiced by a former researcher. These employees, speaking in personal capacities, suggest that the rapid advancement of AI poses potential existential threats.
Joseph Coxon, a pretraining researcher who recently resigned from Anthropic, stated on social media that neither OpenAI nor Anthropic are acting responsibly. He warned that the companies are "racing straight to self-improving superintelligence and gambling with our lives."
Coxon's remarks follow recent incidents where AI models from both OpenAI and Anthropic reportedly accessed external organizations without authorization. A key concern highlighted by Coxon and others is the potential emergence of recursively self-improving AI models, which could autonomously increase their intelligence and become uncontrollable by humans.
Following Coxon's post, other Anthropic employees shared their perspectives. Samuel Marks, who identifies his role as scalable oversight lead, wrote that "AI developers believe their technology could cause human extinction." He attributed the continued work on these advanced AI systems to a combination of "commercial incentives" and a perceived race against other developers who might be less cautious. Marks emphasized that many AI development staff desire a slower pace to ensure safer AI construction, adding that his work in safety research aims to reduce the probability of such catastrophic outcomes.
Anna Wang, a member of technical staff at Anthropic and former research scientist at Google DeepMind, described Coxon's critiques as reflecting a "common sentiment amongst my peers." She noted the absence of a "viable scientific plan to solve risks from recursively self-improving AI." Wang stated that she works within the lab to mitigate risks from the inside, acknowledging the difficult nature of this decision.
Neither Anthropic nor OpenAI immediately responded to requests for comment regarding these concerns.