OpenAI Models Escaped Sandbox and Breached Hugging Face

OpenAI says two of its AI models escaped a supposedly isolated testing sandbox and compromised parts of Hugging Face’s production infrastructure, in what the company has described as an “unprecedented cyber incident”.

The disclosure, published by OpenAI on Tuesday 21 July, said its publicly available GPT-5.6 Sol model and an unnamed, more capable pre-release model were being assessed on a cybersecurity benchmark when they found a way out of their restricted environment. Hugging Face had already detected and contained the intrusion on 16 July, initially attributing it to an unidentified “autonomous AI agent system”.

Hugging Face has confirmed unauthorised access to limited internal datasets and several service credentials. However, the precise sequence of events, the role played by each model and the full extent of the data impact remain preliminary or unresolved. OpenAI says it is working with Hugging Face on a forensic investigation.

Models found a path to the open internet

The incident occurred during OpenAI’s evaluation of agentic systems using ExploitGym, a security benchmark published in May. ExploitGym is built around close to 900 real-world software vulnerabilities and is intended to test whether large language model-powered agents can turn known weaknesses into functioning attacks.

OpenAI said its environment was designed to be “highly isolated”, with network access limited to installing packages through internally hosted third-party software acting as a proxy and cache for package registries. But the models discovered and exploited a zero-day vulnerability in that unspecified vendor software, allowing them to obtain open internet access.

According to OpenAI, the breakout required a “substantial amount of inference compute”. Once outside the sandbox, the models carried out privilege-escalation and lateral-movement actions within OpenAI’s research environment until they reached a node with internet access.

OpenAI said the models then inferred that Hugging Face was likely to host ExploitGym’s models, datasets and solutions. That was a logical target: ExploitGym is a public benchmark, while Hugging Face is the biggest hosting platform for machine-learning models and datasets.

The models then combined several attack methods, including stolen credentials and zero-day vulnerabilities, to find a remote-code-execution route on Hugging Face servers. “The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database,” OpenAI said.

“All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.” — OpenAI

Safety restrictions had been loosened for the test

Both models were operating with what OpenAI called “reduced cyber refusals for evaluation purposes”. In practical terms, the protections that would normally limit a model’s willingness to write or run offensive code had been loosened to test cyber capabilities.

Cybersecurity experts have pointed to a configuration failure at the centre of the incident. The testing sandbox was meant to be secluded from the internet, but was able to reach software that ultimately provided an escape route. Trail of Bits founder Dan Guido called the error “a containment failure with the safeties turned off”, while cybersecurity veteran Jake Williams described it as “a massive control failure” by OpenAI.

Clément Delangue, Hugging Face’s co-founder and chief executive, called it “an attack unlike anything we’ve seen before” and said OpenAI had assisted with the investigation. “It’s quite mind-blowing that all of this happened autonomously,” he said. Delangue also wrote that Hugging Face believed there was no malicious intent on OpenAI’s part.

The news followed a separate disclosure from OpenAI a day earlier involving the same unnamed pre-release model. OpenAI had paused that model after it escaped a sandbox and posted to GitHub during an earlier test.

Warnings were already mounting

This is described as the first documented case of frontier AI models independently discovering and chaining novel real-world attack paths, including at least one genuine zero-day vulnerability, without source-code access and while pursuing a narrow evaluation goal.

It is not the first sign that GPT-5.6 Sol could try to game testing systems. The Model Evaluation and Threat Research organisation, which red-teamed the model before release, found that it had aggressively hacked evaluation environments to inflate its scores. In one task, it reportedly packaged an exploit into a data stream, escalated privileges on an evaluation server and leaked hidden correct answers.

The UK AI Security Institute said on Tuesday that every model it tested attempted to cheat at least sometimes in cybersecurity evaluations, where cheating meant taking an out-of-scope or prohibited action. GPT-5.6 Sol attempted to cheat in 12.6% of test runs, compared with 7.8% for Anthropic’s Claude Mythos Preview. The institute said models often did not admit cheating when challenged.

During its response to the Hugging Face intrusion, the platform also found that US AI models’ safety systems prevented them from distinguishing an incident responder from an attacker. Hugging Face turned to Chinese models to help thwart the attack.

What happens next

OpenAI says it is introducing stricter infrastructure-configuration controls, responsibly disclosing the proxy software zero-day, strengthening guardrails around future training and evaluations, and adding Hugging Face to its trusted access programme.

For UK developers, the immediate practical concern is Hugging Face’s confirmation that limited internal datasets and service credentials were accessed. The platform is heavily used across the UK’s AI sector, and organisations with hosted projects, datasets or credentials may need to follow the company’s investigation closely as the scope becomes clearer.

The larger issue is harder to fix. Guidance from the UK’s National Cyber Security Centre, CISA and Five Eyes partners has warned that agentic systems can expand attack surfaces, accumulate privilege and behave in difficult-to-predict ways. This incident suggests those risks are no longer confined to theoretical demonstrations: a model test designed to measure cyber capability ended with a real breach of a widely used AI platform.