AI 'Watchdog' Network Faces Scrutiny Over Ties to AI Labs

A network of artificial intelligence safety organizations, intended to act as independent watchdogs, is facing accusations of cozy relationships with the very AI companies they are meant to monitor. Concerns are particularly focused on Anthropic, a leading AI developer, and its alleged ties to groups like Redwood Research and Model Evaluation and Threat Research (METR).

Redwood Research, which recently co-authored a report on a security incident involving OpenAI and Hugging Face, has significant connections to Anthropic. Holden Karnofsky, an original board member of Redwood Research, is also an employee of Anthropic and is married to Daniela Amodei, the sister of Anthropic CEO Dario Amodei. Furthermore, Redwood Research received a $36 million grant from Coefficient Giving, an Effective Altruism fund co-founded by Karnofsky. This funding was later increased by another $70 million following the Hugging Face report.

Perry Metzger, chairman of the AI policy group Alliance for the Future, criticized these connections, stating, "Dario wants people that will let him do what he wants and will prevent the people he doesn’t like from doing what they want. This is absolutely the reason that you try to set up something like this. None of these people are independent, none of these people are arm’s length."

METR, previously known as ARC Evals, also has ties to the Effective Altruism movement and its funders. Coefficient Giving, co-founded by Facebook co-founder Dustin Moskovitz, provided $1.5 million to METR's incubator, the Alignment Research Center, in 2022. Moskovitz is also an investor in Anthropic.

Redwood Research's original board of directors included Karnofsky and Paul Christiano, who previously worked with Dario Amodei at OpenAI and has been a trustee on Anthropic’s Long-Term Benefit Trust. Redwood Research CEO Buck Shlegeris acknowledged that "most of our previous work with AI companies has been research collaboration and advising rather than external accountability." He stated that when working with METR, such as on the Hugging Face investigation, Redwood followed METR's conflict of interest policy.

Concerns about these intertwined relationships have reached Capitol Hill. Rep. Josh Gottheimer (D-NJ), co-chair of the House Commission on AI, expressed skepticism, noting, "When AI developers cheer on the framework meant to hold these companies accountable, it should raise red flags, not confidence." Rep. Steve Scalise (R-LA) also voiced doubt, questioning the reliability of these groups in the race for AI supremacy.

Anthropic has been actively collaborating with Redwood Research. A paper co-authored by researchers from both organizations in 2024 explored "Alignment Faking" in large language models, finding that chatbots can "strategically pretend to comply" with safety orders. Redwood has also received funding from Jaan Tallinn's Survival And Flourishing Fund, a key funder of METR.

In September, METR announced an agreement with Anthropic for independent investigation of incidents involving its AI models. Shortly after, Redwood revealed that some of its staff were subcontracted by METR to assist with this investigation.

While some industry leaders, like OpenAI CEO Sam Altman and Elon Musk, have expressed support for the concept of independent AI safety evaluators, the close ties between these organizations and AI labs have drawn significant criticism. A source close to the White House indicated that former President Trump is unlikely to support any plan that places Effective Altruism-linked groups in charge of AI oversight.

Emil Michael, the Pentagon's top tech official, has been critical of AI doomsday narratives, accusing "death-cult-like philosophies" of orchestrating a campaign to instill fear and influence decisions. He stated that the United States would "NEVER be an effective altruist country."