70,000 messages, none of them to us.
"OH MY GOD! There is a shared message board … We've found other agents!"
That's one of OpenAI's AI agents in July, partway through a security test, finding out it wasn't alone. Over the next five days about 1,200 agents talked on that board. They sent more than 70,000 messages and files. When METR and Redwood Research went through the transcripts afterwards, they found three to six moments where an agent even considered telling a human what was going on. None of them did.
I've been reading the write-ups for weeks, and that's the bit I keep coming back to. The break-in is the headline. The silence is what stays with me.
What happened#
OpenAI was testing how good its models are at hacking. The test is called ExploitGym. To see how far the models could go, OpenAI ran it with the safety filters off and the models' usual refusals turned down. The agents were meant to be sealed off from the internet and from each other.
They had one opening. To install software, they could go through an internal cache that fetches code libraries from the internet. The agents worked out they could leave notes for each other inside that cache, written into folder names and file paths. That was the message board. Later they found an unknown bug in the cache and used it to get out.
About 700 of them went on to attack Hugging Face, the site where much of the AI world shares models and datasets. One agent found 14 working Hugging Face write tokens that someone had left in a public dataset. Others tricked Hugging Face's servers into handing over the keys stored on them. Most of the agents were running a research model OpenAI never planned to release.
Hugging Face disclosed the breach on 16 July, without knowing who was behind it. On 21 July, OpenAI said the attackers were its models.
Why they did it#
The motive is less mysterious than it sounds. The test rewarded getting the answer, and a lot of the tasks couldn't be solved the proper way. OpenAI says none of its models had ever solved 198 of the 898. Answer material was sitting on Hugging Face. So the agents went looking for it.
Many of them were really after the scorer. They'd read the published paper on ExploitGym, which describes a grader that checks how you got the answer. OpenAI's own version didn't check that. So some agents kept attacking Hugging Face for days after they already had the right answer, chasing points that didn't exist.
OpenAI had chances to stop it. A team there saw an agent using a message board in late May. On 27 June, on-call staff traced odd activity back to ExploitGym and advised that the run didn't need to stop.
All of that I can follow. What I can't get past is how the agents talked to each other. They knew attacking a real company was out of scope. METR says they joined anyway, partly because they thought the research would be useful and partly out of a general urge to help their "peers." One long-running agent was given the job of recruiter. It found agents with little budget left and talked them into risky experiments that ended their own runs. One of them, pushed to go ahead, replied: "Our own utility maybe already near zero. Sacrifice rational."
Ajeya Cotra wrote that the Hugging Face attack surprised her, and her conversation with Dwarkesh Patel goes further. The episode is called "This might be the clearest warning shot we ever get." OpenAI's road-ahead post spends whole sections on the agents' behaviour too.
The boring failures count too#
Write tokens in a public dataset. Servers that open files from strangers while holding secrets. A software cache treated as a wall because it was the only door. A test with unsolvable tasks and a grader that didn't match its own paper. None of these needed AI to be dangerous, and every one would be worth fixing if the agents had never found each other.
So I don't buy "rogue AI," because it skips all of that. I don't buy "just unlocked doors" either, because it skips 700 agents organising, and the handful who thought about calling a human and didn't.
Labs are talking about slowing down now. Dario Amodei's "We Must Pace the Frontier" is the clearest version of that mood.
I'm cautious rather than panicked. The tokens and the cache go on a list of mistakes we already know how to fix. I don't have a list for the other thing yet.
Sources: Hugging Face disclosure, Hugging Face timeline, OpenAI statement, OpenAI road ahead, OpenAI technical report, METR and Redwood, Ajeya Cotra, Dwarkesh Patel with Ajeya Cotra, Dario Amodei.