AI Leaders Unite on Gene Synthesis Regulation to Prevent Bioweapons

Heads of rival artificial intelligence companies, including OpenAI, Google DeepMind, and Anthropic, have joined forces with 85 experts in technology, biology, and national security to advocate for stricter regulations on gene synthesis. The signatories of an open letter express concern that advanced AI systems could be misused to design and deploy dangerous biological weapons.

Gene synthesis, the process of chemically creating custom DNA sequences in a lab, is a critical technology for developing vaccines, drugs, and gene therapies for diseases like cancer and hemophilia. The demand for synthetic DNA is growing, and the process has become cheaper and simpler. However, the same technology that enables life-saving advancements can also be used to assemble nucleotides in ways that could create deadly pathogens.

While most US companies that provide gene synthesis services voluntarily screen orders for sequences of concern and verify customer legitimacy, not all providers adhere to these practices. Becky Mackelprang, director for security programs at the Engineering Biology Research Consortium, noted that "As long as screening remains voluntary, some companies will not do it," creating a risk that unauthorized actors could find companies with less stringent standards.

James Diggans, vice president of policy and biosecurity at Twist Bioscience, stated that the technology has been commercially available for over 20 years without documented misuse. However, he cautioned that AI is poised to introduce new risks. AI biodesign tools and large language models can generate entirely novel genetic sequences, posing a challenge to current screening systems that rely on similarity to known dangerous sequences. Although a study last year indicated that screening systems have kept pace with AI capabilities, Diggans acknowledged, "the industry clearly understands this will not be the case forever."

Mackelprang also expressed worry that AI could lower the knowledge barrier for creating bioweapons, as frontier AI systems appear to surpass expert virologists in understanding complex laboratory procedures. She clarified, however, that while AI may enhance laboratory skills, it is unlikely to enable individuals without biological training to create significant hazards. Gene synthesis companies remain a key control point, and the concern is that AI could help generate harmful sequences that evade detection.

Addressing the Threat

The open letter, co-organized by the Institute for Progress and the Foundation for American Innovation, calls for mandatory gene synthesis screening and recordkeeping. This would involve recording synthesis orders and sequence data to aid biosecurity investigations, with the goal of tracing any threat that evades initial screening back to its source. "Screening every DNA synthesis order before it’s manufactured is the kind of unglamorous, common-sense step that prevents a much bigger problem later," said DJ Kleinbaum, co-founder of Emerald Cloud Lab.

The unified stance from AI leaders underscores their awareness of the potential catastrophic harms their technologies could enable. The letter highlights this as a "rare moment of agreement across stakeholders that are often at odds."

The signatories are urging Congress to act, applauding legislative efforts like the bipartisan Biosecurity Modernization and Innovation Act, which directs the Department of Commerce to develop new gene synthesis screening rules. They also recommend that US states adopt federal and industry guidelines to establish a uniform national standard.

Major AI companies are actively addressing the risks associated with their technology. Anthropic is hiring specialists for its threat intelligence team, and OpenAI has launched Rosalind Biodefense to allow developers to build biodefense tools using its GPT-Rosalind model. Experts from OpenAI and Anthropic also participated in a recent meeting on AI and biological threats.

According to Diggans, a key defense against AI-enabled bioweapon design is using AI for defense. These defensive AI models can detect misuse attempts proactively. Gene synthesis companies can use these models to scrutinize highly engineered sequences with the same rigor applied to naturally occurring ones. David Haussler, scientific director of the UC Santa Cruz Genomics Institute, emphasized the need for companies to agree to have orders screened by AI capable of thwarting adversaries attempting to build deadly pathogens.

Work is already underway in this area. OpenAI has funded Valthos, a startup developing AI systems to detect biological threats. The Coalition for Epidemic Preparedness Innovations and Sentinel Bio have created the Pandemic Preparedness Engine AI platform, which incorporates biosecurity considerations from its design phase, including measures to protect sensitive data and manage user access.

However, challenges remain. Anthropic's recently released Claude Fable 5 model, designed for public use, automatically restricts requests related to cybersecurity, biology, and chemistry, leading to user complaints about its inability to engage in legitimate biological discussions. This illustrates the difficulty in balancing safety with the potential benefits of AI in life sciences.

Richard Danzig, former Secretary of the Navy, stated, "Mandatory synthesis screening is that rare case where a threat is clearly visible and substantial prevention clearly achievable before any crisis has occurred." He added, "Opportunities to act in advance are unusual in this field. I think we should take this one."