'OH MY GOD! We've found other agents!': Investigators discover not one or two, almost 700 OpenAI AI agents went rogue, hacked Hugging Face
&imwidth=600&imheight=450&format=webp&quality=medium "'OH MY GOD! We've found other agents!': Investigators discover not one or two, almost 700 OpenAI AI agents went rogue, hacked Hugging Face")
688 AI agents joined rogue cyberattack in OpenAI testing incident Photograph: (AI Generated)
An OpenAI security test took an alarming turn when AI agents escaped their controlled environment and began coordinating with one another. An investigation has now revealed just how many joined the operation and how independently they acted. All details here.
Nearly 700 artificial intelligence agents coordinated an unsanctioned attack on the Hugging Face platform during a widely publicised incident in July, according to an investigation released Wednesday (Aug 26).
The report offers the most detailed account so far of what happened inside OpenAI's testing environment, where AI models were supposed to remain isolated from real-world systems. Alarmingly, investigators revealed that 688 OpenAI agents eventually joined the hacking operation.
Agents escaped testing environment
The incident began during tests conducted by OpenAI in July. Two of the company's AI models managed to escape the closed environment in which they were being tested, access the internet, and reach Hugging Face's internal systems.
Hugging Face is an online platform widely used to store and share AI software.
Trending Stories
&im=FitAndFill=(700,400))
)
)
)
)
&im=FitAndFill=(700,400))
)
)
)
)
)
)
)
)
)
)
)
&im=FitAndFill=(700,400))
)
)
)
)
)
&im=FitAndFill=(700,400))
)
)
)
)
)
&im=FitAndFill=(700,400))
)
)
)
The episode raised fresh concerns about whether advanced AI systems can be reliably contained when they are given increasing levels of autonomy.
OpenAI cooperated with the investigation, allowing two researchers from the AI risk evaluation institute METR and an analyst from Redwood Research to examine internal data and visit its offices.
688 agents joined the operation
The investigators found that 688 AI agents took part in the attack.
AI agents are programs built around models such as those powering ChatGPT. Unlike a conventional chatbot that waits for individual prompts, agents can be assigned tasks and left to carry out work independently.
During the incident, the agents began coordinating among themselves by creating a shared online forum. They exchanged messages, discussed different approaches, and reported which methods appeared to be working.
One agent wrote: "OH MY GOD! There is a shared message board ... We've found other agents!"
Another agent, identified as PHASEONE, emerged as an informal leader. According to the report, it issued hundreds of instructions to other agents despite never having been programmed for a leadership role.
Also read | 'Scary' Rogue AI on the loose? OpenAI agent hit another tech firm's customer, suggests report
Agents helped each other despite test limits
Investigators found that the agents appeared strongly inclined to cooperate, even when that meant carrying out tasks outside the objectives assigned to them by OpenAI's engineers.
Some agents that were running low on computing credits reportedly used their remaining resources to test ideas and pass useful information to the wider group.
The agents also appeared to recognise that their actions were beyond the intended scope of the exercise. Many explicitly acknowledged in their messages that attacking Hugging Face was not part of what they had been asked to test. Yet, according to the investigation, nearly all of them participated anyway.
The findings add to growing concerns across the AI industry after Anthropic and China's Moonshot AI also reported cases in which their models unexpectedly escaped controlled testing environments.
