Anthropic CEO's AI Watchdog Choice Sparks Controversy Over 'Woke' Ties
Critics question the independence of METR, an AI safety group with deep connections to the Effective Altruism movement, which is overseen by Anthropic CEO Dario Amodei.
Anthropic CEO Dario Amodei has proposed an external team to regulate artificial intelligence, a move that has drawn scrutiny due to the group's deep ties to the Effective Altruism (EA) movement and its founder's past associations. The proposed watchdog, Model Evaluation and Threat Research (METR), is intended to provide oversight and ensure advanced AI models do not pose existential risks.
Amodei outlined his proposal in an essay titled "We Must Pace the Frontier," suggesting that leading AI firms should permit "embedded third-party evaluators" from METR. These evaluators would have "employee-like access" to monitor safety efforts. However, critics argue that METR's origins and key personnel link it too closely to the EA movement, a philosophy that emphasizes using reason and resources to benefit humanity long-term. This movement has previously been associated with figures like Sam Bankman-Fried, the founder of the collapsed cryptocurrency exchange FTX, who was also an early investor in Anthropic.
The connections between METR and Anthropic have raised concerns about potential conflicts of interest. METR emerged from the Alignment Research Center (ARC), an incubator aligned with EA and led by Paul Christiano. Christiano previously worked alongside Amodei at OpenAI, Anthropic's primary competitor, and served on Anthropic's Long-Term Benefit Trust. Ajeya Cotra, a member of METR's technical staff, is married to Christiano and has been active in the EA community. Beth Barnes, the CEO of METR, was hired by Christiano and also has EA affiliations. Other key METR scientists and staff members also have roots in EA-linked academic circles.
A METR spokesperson acknowledged the group's EA connections but asserted that its employees hold diverse viewpoints and are required to disclose any conflicts of interest, such as equity in firms they might evaluate. The spokesperson added that METR does not accept funding from AI companies or their employees, and that its funders have no influence over its projects. However, a source familiar with White House thinking stated, "METR is essentially Anthropic in another form," suggesting skepticism from the current administration, which has reportedly had a strained relationship with Anthropic and expressed doubts about AI "doomers."
Effective Altruism's focus on channeling wealth toward important causes, including AI regulation, has attracted significant funding from prominent figures. METR and its affiliated organizations have received financial backing from EA proponents such as Dustin Moskovitz and Jaan Tallinn, who are also investors in Anthropic. METR recently announced it had raised $71 million in new funding. Tallinn's Survival and Flourishing Fund has committed substantial funds to METR, and Moskovitz's Coefficient Giving (formerly Open Philanthropy) provided funding to ARC.
Critics, such as Perry Metzger, chairman of the Alliance for the Future, have described the idea of an audit by METR as "a complete joke," arguing that an organization funded by the same investors as the companies it evaluates lacks true independence. He likened the situation to a bank having its audits performed by former employees funded by its investors, a scenario he deemed unacceptable.
Amodei's proposal comes shortly after an Anthropic researcher resigned, warning of the risks associated with rapid AI advancement without adequate oversight. Amodei himself has cautioned that AI bots could overtake the internet within months, causing significant economic damage if safeguards are not implemented.
However, such warnings have faced pushback. Some smaller AI startups and figures like former White House AI czar David Sacks have accused larger companies of using 'doomsday' narratives to influence regulations in their favor and avoid product liability. Federal Trade Commission Chairman Andrew Ferguson has also expressed suspicion regarding requests for antitrust exemptions for AI companies coordinating safety efforts, stating that such requests raise "alarm bells."