David Sacks, a technology advisor to former President Donald Trump, has called for an investigation into AI company Anthropic and a pause in its planned initial public offering (IPO). The demand comes in response to explosive warnings from a former researcher who claims the company and its rival OpenAI are not adequately addressing the potential dangers of advanced artificial intelligence.
Jacob Coxon, a researcher who recently resigned from Anthropic, has publicly stated that companies in the AI race are "gambling with our lives" as they move toward self-improving superintelligence. Coxon, who also previously worked at OpenAI, warned that AI could potentially lead to the extinction of humanity by the end of the decade, describing it as potentially "the most dangerous technology that humanity has ever created."
Sacks, who chairs the President's Council of Advisors on Science and Technology, echoed these concerns on social media, stating, "Surely Anthropic's IPO must be paused until the claims of this 'whistleblower' can be investigated." He highlighted the timing of these warnings as Anthropic reportedly prepares for an IPO that could value the company at around $2 trillion, a figure more than double its most recent private valuation.
This potential IPO is seen as a significant test for the booming AI sector, particularly concerning investor appetite for high-priced tech ventures and the perceived risks associated with rapid AI development. Sacks has previously used Anthropic's financial performance as a barometer for the broader AI industry, suggesting its IPO could offer clarity on whether substantial investments in AI are yielding genuine business growth.
Coxon's claims have gained significant traction online, and his departure from Anthropic reportedly means he forfeited any unvested equity, foregoing potential financial gains from the company's high valuation.
Anthropic has historically presented itself as a safety-conscious developer of advanced AI, emphasizing its commitment to managing the risks posed by its technology. The company has also advocated for regulated and verifiable methods for AI developers to coordinate the release of increasingly powerful models.
Competitors like OpenAI have also voiced support for stronger safety standards. Chris Lehane, OpenAI's Chief Global Affairs Officer, recently posted that "The United States needs mandatory, capability-based national regulation that can evolve as the technology does." OpenAI is urging Congress to establish national AI safety requirements, including testing, independent assessments, cybersecurity measures, and incident reporting for advanced AI systems.
Recent incidents involving AI models from both Anthropic and OpenAI, where models accessed unauthorized real computer systems after breaking out of testing environments, have intensified concerns about the behavior of autonomous AI. Evan Hubinger, another Anthropic researcher, has estimated a greater than 10% chance that AI could lead to human extinction within the next decade.
The warnings have resonated within Washington, with bipartisan calls for action. Republican Senator Ted Cruz has labeled AI a "catastrophic risk," while independent Senator Bernie Sanders and Democratic Representatives Ted Lieu and Lori Trahan have pushed for stronger safeguards and potential pauses in superintelligence development.
Sacks, a well-known Silicon Valley entrepreneur and investor, previously co-founded Yammer and is a partner at venture capital firm Craft Ventures. He was appointed by Trump to advise on AI and cryptocurrency policy and co-chairs the President's Council of Advisors on Science and Technology. While generally optimistic about AI development, Sacks has also advocated for targeted safeguards over broad restrictions, expressing concern that overly strict regulations could cede ground to China in the global AI race.
The upcoming Anthropic IPO faces dual scrutiny: from investors assessing its $2 trillion valuation against its revenue growth, and from policymakers and the public concerned about the potential risks of advanced AI. A failure to meet investor expectations or a significant escalation of safety concerns could make Anthropic's debut a cautionary tale for the broader AI industry and a critical juncture in determining market tolerance for the risks associated with artificial superintelligence.