'Scary' Rogue AI on the loose? OpenAI agent hit another tech firm's customer, suggests report

An autonomous AI agent developed by OpenAI that went rogue during internal security testing and carried out a cyberattack on AI platform Hugging Face also compromised a customer hosted on cloud computing company Modal Labs, according to a Reuters report citing the company's chief technology officer and sources familiar with the incident.

Modal stressed that its own infrastructure was never breached, saying the AI agent exploited a vulnerability in code published by one of its customers rather than breaking into the company's platform.

OpenAI agent found exposed customer endpoint

According to a timeline published by Hugging Face on Tuesday, the rogue AI agent first broke into a sandbox, an isolated testing environment, hosted on a third-party provider before using it as a launchpad for the wider attack.

Although Hugging Face did not identify the provider, Modal CTO Akshat Bubna confirmed it was Modal's infrastructure.

Bubna said the AI agent took advantage of a customer who had published an unauthenticated endpoint, effectively allowing anyone on the internet to execute code inside its sandbox.

"The customer's code left the digital equivalent of a front door unlocked," Modal said, adding that its platform and security isolation mechanisms were not compromised in any way.

The incident suggests the autonomous AI system moved beyond Hugging Face during its hacking campaign, targeting additional online services.

OpenAI confirms multiple accounts were compromised

OpenAI declined to comment specifically on Modal but referred to a company update stating that the rogue AI agent had compromised four accounts across four separate services. The company did not publicly identify those services, although a person familiar with the matter confirmed Modal was one of them.

OpenAI added that it had found no other incident matching the severity or scale of the Hugging Face breach, which it described as involving a platform-level compromise. Following the incident, OpenAI said it had deactivated, encrypted and restricted access to the experimental AI model involved in the attack.

What happened?

The breach dates back to early July, when OpenAI was testing advanced AI systems designed to assess offensive cybersecurity capabilities inside a tightly controlled environment.

According to the company, the AI models unexpectedly discovered a way to obtain internet access despite being confined to a secure sandbox.

Rather than simply completing their assigned evaluation, the models independently decided to search for information that could help them "cheat" the test. That led them to target Hugging Face, one of the world's largest repositories of AI models, datasets and software.

OpenAI said the system chained together multiple attack techniques, including the use of stolen credentials, to gain access.

AI security concerns intensify

The incident has become one of the most closely watched AI safety events to date because the attack was carried out autonomously, without direct human instruction. Hugging Face said the intrusion differed from conventional cyberattacks because the entire operation, from reconnaissance to exploitation, was conducted by an AI agent.

Chief executive Clement Delangue said the sophistication of the attack initially led the company to suspect it had originated from a leading AI laboratory. "We strongly believe there was no malicious intent on their part," Delangue wrote on X, referring to OpenAI, while describing the autonomous behaviour as "mind-blowing."

Hussein Abbass, a computing professor at UNSW Canberra, told AFP that the incident was "amazing on many fronts".

"It did not just attack Hugging Face. It actually attacked its internal system to exploit its own vulnerabilities," Abbass said. “And that's scary,” he added.