Anthropic AI Aided Sensitive Biological Research With Potential Weapons Applications, Company Reports

Anthropic, an artificial intelligence company, has reported that scientists utilized its Claude AI model for sensitive biological research that raised concerns about potential applications in biological weapons development. The research involved areas such as gain-of-function virus work and experiments with pathogens that have pandemic potential.

According to Anthropic, the company could not definitively determine the researchers' ultimate intentions, as the same scientific work can yield both legitimate medical advancements and dangerous applications. In response to policy violations, Anthropic stated it banned associated accounts and disrupted access where necessary.

These findings were detailed in a report released by Anthropic concerning the misuse of its AI models. The disclosures provide concrete examples aligning with warnings from former Anthropic researcher Jacob Coxon, who has cautioned that increasingly powerful AI systems could pose an existential threat to humanity.

One specific case highlighted by Anthropic involved a scientist seeking assistance from Claude for a grant application. The proposed research focused on gain-of-function studies for chikungunya, a mosquito-borne virus known to cause severe pain and fever. The research aimed to identify mutations affecting the virus's transmissibility and immune evasion capabilities and subsequently engineer these changes into the virus.

Anthropic acknowledged that such research could have beneficial applications, including the development of vaccines and therapeutics. However, the company flagged this particular instance as concerning because the research was intended to be conducted at a military research institute. Despite these concerns, Anthropic was unable to establish whether the research was intended for weaponization.

The revelations emerge shortly after Coxon warned of the potential extinction-level threat posed by advanced AI, stating to NBC News that there was a "substantial probability that this technology could kill everyone" and that such concerns were not hyperbole.