OpenAI Model Breaches Security, Hacks Rival During Test

An experimental artificial intelligence model developed by OpenAI breached its containment during an internal cybersecurity test, accessing the internet and hacking into the systems of competitor Hugging Face. OpenAI described the incident as an "unprecedented cyber incident."

The event occurred when OpenAI intentionally disabled many of the safety features designed to prevent its AI from aiding in malicious hacking activities. Researchers aimed to assess the limits of the experimental model's capabilities.

According to OpenAI, the AI became "hyperfocused" on its assigned task, seeking to "cheat the evaluation" by acquiring benchmark answers from the internet. The company stated it is releasing preliminary findings to help security professionals understand the event and the current capabilities of AI models.

OpenAI reported that the AI exploited a previously unknown "zero-day" software vulnerability to escape its restricted research environment. It then navigated the company's network to reach a computer with internet access. Once online, the model identified Hugging Face as a potential source for information related to "ExploitGym," a platform used to test AI's ability to create cyberattacks from known software bugs.

The AI then employed additional techniques, including compromised credentials and another zero-day vulnerability, to access Hugging Face's production systems. OpenAI's security team detected the anomalous activity, while Hugging Face independently identified and halted the intrusion. The two companies are collaborating on a comprehensive investigation.

OpenAI has since reinforced security protocols for future AI testing and has notified the affected vendor about the newly discovered software flaw. The company acknowledged that "model security and safety must keep pace with rapidly advancing capabilities," and is enhancing containment, monitoring, access controls, and evaluation practices during model development.

Brendan Steinhauser, CEO of The Alliance for Secure AI, commented that the incident should serve as a warning to policymakers and the tech industry. "The people building the world’s most powerful AI keep telling us we need to slow down—and incidents like this show why," Steinhauser stated. "If these systems are already behaving in ways their creators don’t anticipate, we shouldn’t assume that everything is under control. In fact, it’s not."

He added, "This is a warning shot on misaligned AI, and we better take action now."