Artificial intelligence company Anthropic announced it has intercepted several attempts this year to use its AI models for biological weapons development. The company's AI misuse report, obtained by The New York Times, indicates that researchers were attempting to conduct research that could lead to the creation of dangerous pathogens.
Anthropic stated it could not definitively determine if the research served a legitimate or nefarious purpose, but chose to err on the side of caution due to the dual-use nature of biological inquiry. The report detailed how scientists allegedly circumvented controls to access AI models from restricted regions and worked to obscure the intent of their research to bypass safeguards.
One specific instance involved a scientist requesting assistance from Anthropic's chatbot, Claude, to draft a grant application for research on the chikungunya virus. While research on the virus can contribute to legitimate advancements like vaccines, Anthropic noted it could also be used to engineer a more dangerous version of the disease. This particular case was deemed concerning as the proposed work was slated to occur at a military research institute.
Jacob Klein, head of threat intelligence at Anthropic, told The New York Times that the possibility of weaponization at a military institution conducting gain-of-function research is a significant concern. Andrew Weber, a senior fellow at the Council on Strategic Risks, echoed these sentiments, calling the findings "chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities" of AI, particularly given the ongoing development of prohibited biological weapons by countries like Russia, China, and North Korea.
Beyond biological threats, Anthropic's report also identified attempts by entities in China, Russia, and Yemen to use Claude for developing software to design firearms, missiles, armed drones, and bombs. The report also noted alleged attempts by suspected Chinese and Iranian government actors to target dissident and diaspora communities for surveillance, and the use of Claude by Russian state media to generate election propaganda, including fabricated claims about Moldova's parliamentary election.
These revelations follow high-profile warnings from Jacob Coxon, a former researcher at Anthropic, who resigned this week accusing the company and rival OpenAI of not adequately addressing the potential dangers of AI. Coxon expressed fears that researchers believe AI could lead to the extinction of humanity, stating that companies are "racing straight to self-improving superintelligence and gambling with our lives."
Coxon's comments have gained significant traction online, prompting renewed scrutiny of the AI industry. David Sacks, chair of the President's Council of Advisors on Science and Technology, has called for Anthropic's planned initial public offering (IPO) to be paused until Coxon's claims can be thoroughly investigated. Coxon described AI as potentially "the most dangerous technology that humanity has ever created," warning that the pace of AI development might outstrip safety measures.
Anthropic has historically positioned itself as a safety-conscious AI developer, advocating for secure and verifiable methods for coordinating the release of powerful AI models. Competitor OpenAI has also voiced support for stronger safety standards in the field.