Artificial intelligence company Anthropic announced Thursday that it has prevented multiple attempts this year by scientists aiming to use its AI models to research the development of biological weapons. The company’s findings are detailed in a new report outlining instances of AI misuse.
Anthropic presented five case studies where individuals allegedly circumvented safety controls to engage in research that could contribute to bioweapon development. These actors reportedly bypassed restrictions on access for unsupported regions and used methods to obscure their research objectives to evade safeguards.
"When we detected and investigated these cases, we banned the users’ accounts and incorporated our investigative findings into our frontier model safeguards, enforcement and threat intelligence processes to better prevent, detect and disrupt these activities in the future," Anthropic stated in its report. The company did not disclose the names of the research institutions, their countries, or the specific biological agents involved.
Anthropic noted that distinguishing between legitimate scientific inquiry and malicious intent can be challenging, as research leading to medical advancements can also produce dangerous pathogens. The report highlighted the dual-use nature of biological research, which sophisticated actors can exploit to maintain "plausible deniability" about their intentions.
"Sophisticated actors are aware that we … are attempting to detect dangerous uses of our models, and they use the dual-use nature of biology to maintain a kind of ‘plausible deniability’ about their research," the report stated. "More sophisticated actors can hide their intent, extracting assistance from an AI model in interactions that look plausibly beneficial, but when put in context and analyzed holistically, can provide clear warning signs of misuse."
One case study involved a scientist in May who sought assistance from Anthropic's AI chatbot, Claude, for a grant application. The research proposal focused on engineering mutations to make the mosquito-borne chikungunya virus more harmful, despite the potential for such work to lead to vaccine development. Anthropic flagged this instance due to concerns about the affiliated institution, which was identified as a military research institute, even though the application suggested civilian researchers were involved.
Beyond biological threats, Anthropic's report also cited misuse of its AI models in cyber operations, surveillance, and the construction of conventional weapons. The potential for AI to be used in biological warfare has become a significant concern among experts, even as public attention often focuses on cybersecurity risks and AI alignment issues.
Andrew Weber, a senior fellow at the Council on Strategic Risks, described Anthropic's case studies as "chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities" of leading AI models. The report's release coincides with internal discussions at Anthropic regarding AI safety and existential risks, following warnings from a former researcher about the company's trajectory toward advanced AI.