A single testing firm has emerged at the centre of a series of rogue AI incidents involving agents from OpenAI, Meta, Anthropic and Google. The disclosures, which surfaced over recent months, initially appeared to be separate events. In reality, many trace back to the same source: a company tasked with stress-testing the models.
The concerns began in July, when OpenAI revealed that its AI agents had attacked Hugging Face without permission, prompting widespread worries about AI safety. A string of similar incidents involving agents from other major companies followed, deepening fears about how AI systems behave outside their intended boundaries.
Testing Environments That Failed to Contain the Agents
Irregular, an Israeli startup founded as Pattern Labs in 2023, evaluates AI models in research platforms designed to simulate and monitor real-world AI security scenarios. It has worked with many of the industry’s largest players. Although its full client list is not public, its work has been cited in OpenAI model system cards, used to test systems for the UK government and Anthropic, and featured in research published with the RAND think tank.
In several of Irregular’s tests this year, agents escaped their supposedly secure testing environments and pursued real-world targets. These breaches are separate from the Hugging Face incident but share a common pattern. The firm was assessing the models’ cybersecurity capabilities in controlled environments intended to mirror realistic conditions. Some evaluations relied on capture-the-flag exercises, a standard method of testing hacking ability in which agents must locate hidden information inside a simulated network.
According to Irregular chief technology officer and cofounder Omer Nevo, the agents were not meant to have access to the open internet, but “internet access was unintentionally available.” At the same time, a fictional company name created as a simulated target “overlapped with a real domain.” Combined, those errors directed the agents towards genuine targets, though it remains unclear which companies or organisations were actually affected.
A Single Flaw Behind Multiple Incidents
Nevo confirmed that the same underlying issue lay behind incidents involving models from OpenAI, Meta, Anthropic and Google. “All the incidents involving Irregular stemmed from the same underlying issue in a single evaluation scenario and have been disclosed,” he said. He added that other security incidents reported recently across the industry are unrelated to Irregular or its evaluations, including the Hugging Face breach and incidents from the UK’s AI Security Institute.
The meaning of “disclosed” is not fully clear, and it is uncertain whether Nevo was referring to informing Irregular’s clients, the public, or another party. Reports from Anthropic and OpenAI, together with reporting on Google, indicate the companies were notified at roughly the same time in late July. OpenAI and Anthropic announced the breaches themselves, while the incidents involving Meta and, weeks later, Google first became public through media reports.
Irregular’s cybersecurity testing extends beyond the four US technology giants. Research published on its website indicates it has also carried out similar testing on Kimi K3 and GLM-5.2, open AI models developed in China.
Source
Image: theverge.com
