The OpenAI lab leak was more extensive than we thought | CNN Business

An OpenAI test that escaped its cage and alarmed the AI and cybersecurity industry attacked more than just Hugging Face, the AI platform that initially appeared to be the sole victim of the virtual lab leak.

An OpenAI test that escaped its cage and alarmed the AI and cybersecurity industry attacked more than just Hugging Face, the AI platform that initially appeared to be the sole victim of the virtual lab leak.

OpenAI, in an update about its ongoing investigation of the incident, said its rogue agent had also broken into several publicly available services and accounts.

The test of OpenAI’s models was supposed to take place in a digital sandbox, a supposedly inescapable lab environment that allows researchers to roll back safety barriers to discover the tool’s maximum hacking power.

But the AI agents, determined to ace the cybersecurity test, broke out of the sandbox and gained access to the real internet. It then hacked Hugging Face to find the answers to the test.

To get into Hugging Face’s servers, OpenAI’s rogue agents needed to find tools around the internet that were necessary for them to break in. OpenAI says the agents found several public-facing websites, including pages that share code, web utilities, screenshots and other information, to help create the code needed to hack Hugging Face. OpenAI did not disclose the other sites its agents attacked.

The agents uncovered leaked usernames and passwords to four accounts across multiple online services and used those credentials to gain access to them. One compromised account was likely used to disguise the AI so it could appear legitimate and bypass Hugging Face’s security protocols. Another was used to store the data the agents were stealing, OpenAI says.

For the two other compromised accounts, OpenAI agents accessed and read the information but did not alter it.

OpenAI said none of the other hacked sites reached the same level of access as what its agents accomplished with Hugging Face.

Essentially, OpenAI said its agents created an Ocean’s Eleven-like heist. Instead of just taking the test, it developed a master plan to break out of its jail, find the keys to the safe, build a safe house and hire a getaway car before it stole the goods.

But OpenAI never told the agents to hack Hugging Face. They performed that action on their own to steal the answer key to the test it was given. The test model figured that cheating was the easiest path to success – even if it meant daisy-chaining attacks together.

Hugging Face CEO Clem Delangue called the nature of the breach “unprecedented.”

“TL;DR: An AI agent escaped its sandbox, cheated on its benchmark test, and hacked our infrastructure to steal the answer key,” Hugging Face said in a blog post detailing the incident.

The good news is that while the breach is a significant moment in AI and cybersecurity, the damage wasn’t significant. Hugging Face said the only customer data that the rogue agents accessed were some search queries used to steal a set of challenge solutions stored across several company datasets. OpenAI’s agents never compromised or accessed any customer-facing models or data, the company said.

OpenAI said it continues to investigate the incident and will make recommendations about how to avoid similar problems in the future once it better understands the scope of the breach.

“We take our responsibility to identify and prepare for risks from increasingly capable AI systems seriously,” the company said. “Once we complete our review, we will review with the Safety and Security Committee and Safety Advisory Group under our Preparedness Framework⁠.”