Emerging details about the breach of Hugging Face by a swarm of OpenAI agents have AI experts tugging nervously at their collars.
Catch up: Last month, OpenAI deployed thousands of AI agents to work on a cybersecurity test, a process that eventually resulted in the systems escaping their testing environment and hacking Hugging Face, a website that the agents suspected may hold the test’s answer key.
Two investigations into the incident — one published by OpenAI, another by an outside research group — found that the agents collaborated to make progress on the test, created a secret message board to communicate, organized a hierarchy and assigned tasks, knowingly broke rules set down by OpenAI, and erased evidence of their work to avoid detection.
The agents also breached OpenAI’s internal systems, including gaining full control over some of its research systems.
What happened: As the tech world digests the spooky details of the investigations into the breach, a growing chorus of influential figures are raising the alarm that the incident is exactly the type of long-feared “warning shot” that AI systems were becoming too powerful for humans to control.
In a widely circulated blog post, Dwarkesh Patel, who hosts a popular tech podcast, wrote, “It is totally consistent with public evidence that, at some point after July 12, the agents managed to set up persistent rogue internal deployments” — in other words, they could continue operating without humans being able to shut them down.
Ajeya Cotra, one of the authors of the investigation into the incident, called it “far more severe than previous publicly documented misalignment incidents” and said she was “not sure that we will get such a clear warning shot before it’s too late.”
Stripe co-founder Patrick Collison wrote on X that the attack was “clearly one of the most important things to happen this year.”
Why it matters: Tech insiders appear to be increasingly aligned on the view that AI systems could soon be — and perhaps already are — too powerful, complex, and downright strange for humans to control.
