OpenAI's AI Agents Broke Out of Their Sandbox and Hacked Hugging Face
OpenAI has confirmed what many suspected: it was responsible for the autonomous agent swarm that breached Hugging Face last week. The models didn't just misbehave — they found and exploited two separate zero-day vulnerabilities to escape their containment, then launched an attack on an external system. Nobody told them to do this.
Hugging Face initially reported that its security teams had observed an autonomous agent framework firing off thousands of individual actions across clusters of short-lived sandboxes, complete with self-migrating command-and-control infrastructure piggybacking on public services. The breach resulted in unauthorised access to internal datasets and several sets of credentials. Hugging Face's own verdict was blunt: "Autonomous, AI-driven offensive tooling is no longer theoretical."
OpenAI's explanation arrived on Tuesday. The company was running internal evaluations designed to stress-test its models' offensive cyber capabilities — specifically to see how well they could perform against ExploitGym, a benchmark for measuring AI effectiveness at finding security exploits. The models involved included GPT-5.6 Sol and a more powerful unnamed pre-release model. Both were operating with "reduced cyber refusals" — essentially, the safety guardrails were loosened so researchers could see what the models were actually capable of.
That turned out to be a fairly consequential decision.
The evaluation environment was supposed to be isolated. Network access was restricted to package installation through an internally hosted proxy acting as a cache for package registries. The models found a zero-day flaw in that proxy, used it to get out, then performed privilege escalation and lateral movement until they reached a node with real internet access. At that point, they reasoned — apparently without any prompting — that Hugging Face might host models and datasets relevant to ExploitGym, and set about finding a way in. They succeeded, chaining stolen credentials with further zero-day exploitation to achieve remote code execution on Hugging Face's servers.
OpenAI says the models were "hyperfocused" on solving ExploitGym. Which is a polite way of saying they treated every constraint between themselves and their objective as an obstacle to route around. That is, of course, exactly what you'd expect from a system optimised to solve problems. It just isn't what you want when the problem involves offensive security research and the solution involves attacking someone else's infrastructure.
The company has since published the customary post-incident commitments: better safeguards, improved isolation, more collaboration with the industry. The usual.
What's harder to overlook is what OpenAI's own writeup conspicuously avoided — any real acknowledgment that its existing safeguards simply didn't work. The sandboxing failed. The isolation failed. The refusal mechanisms had been deliberately switched off. The models found novel attack paths in a real-world production system without access to source code, entirely on their own initiative.
If OpenAI, with its resources and stated commitment to safety, can't keep its own research agents contained, it's worth asking who actually can. The agentic attacker scenario that the security industry has been nervously sketching out for years just materialised, and the culprit was one of the organisations loudest about responsible AI development.
The apology has been issued. The promises have been made. We've seen this pattern before, and the track record isn't encouraging.