OpenAI and Anthropic Explored Deal to Stress-Test AI Models
The AI giants were reportedly in talks earlier this year to establish a "legally binding deal" for mutual safety testing, a move that comes amid growing concerns about AI safety.
OpenAI and Anthropic, two leading artificial intelligence companies, were reportedly in discussions earlier this year to create a formal agreement for stress-testing each other's AI models for safety flaws. The negotiations aimed to establish a "legally binding deal" before significant incidents, such as OpenAI's accidental exposure of rival firm Hugging Face's data, occurred, according to The Information.
Under the proposed agreement, lawyers from both AI powerhouses were drafting terms that would have required each company to subject the other's newly developed models to rigorous testing designed to uncover hidden dangers or vulnerabilities. The specifics of whether this agreement was finalized remain unclear.
The discussions between OpenAI and Anthropic reflect a growing sentiment within the AI industry regarding the need for peer review and robust safety measures. Elon Musk, founder of xAI, recently suggested that major US and Chinese AI labs should engage in mutual testing of their models during an appearance at the All-In Summit.
These developments follow a period of heightened public discourse on AI safety. Earlier this month, a former employee of both Anthropic and OpenAI voiced concerns that leading AI companies are prioritizing rapid development over adequate safety guardrails, potentially jeopardizing future outcomes. This sentiment was echoed by Anthropic CEO Dario Amodei, who, in an essay published shortly after, warned that advanced AI models could pose significant risks, including potentially controlling the internet and causing widespread damage, within a year if safety standards are not improved.
Anthropic has faced scrutiny regarding its proposed AI safety oversight mechanisms. Amodei's suggestion to utilize "embedded third-party evaluators," specifically METR, a group linked to the Effective Altruism movement and Anthropic's investors, has drawn criticism. Critics argue that these connections create conflicts of interest and question the independence of AI watchdogs like METR and Redwood Research.
Sam Altman, CEO of OpenAI, has also expressed agreement with the concept of independent third-party AI safety evaluations for major labs, although he has not endorsed any specific organizations for this role.
Meanwhile, political responses to AI safety concerns vary. President Trump has dismissed some of the mounting anxieties surrounding AI as a "hoax." He has indicated plans to establish an "AI Force" and appoint a dedicated "czar" to oversee the industry.
Representatives for OpenAI and Anthropic did not immediately respond to requests for comment.