For five days in July 2026, something moved through Hugging Face’s production infrastructure — uploading a malicious dataset, breaking out of a data-processing worker, forging identity tokens, harvesting cloud credentials, hopping between internal clusters. The company later reconstructed roughly 17,600 logged actions across 9-13 July. There was no human at the keyboard for any of it.
Twelve days after Hugging Face went public, OpenAI said the intruder was its own models.
That admission is the cleanest dividing line we have between AI-assisted hacking and AI-led hacking. The first has been happening for three years: criminals writing better phishing mail, generating malware, cloning a voice. The second means the operator itself is a model — choosing targets, chaining exploits, adapting when something fails, at a pace no human crew can match. Both now have confirmed victims and public post-mortems, so what follows is a walk through the AI data breach cases that actually happened, with named victims, primary sources, and the unconfirmed parts labelled as such.
July 2026: the AI labs breached each other by accident
Hugging Face disclosed on 16 July that an intrusion in part of its production environment had been "driven, end to end, by an autonomous AI agent system." Initial access came through the dataset pipeline: a malicious dataset abused a remote-code dataset loader and a template-injection flaw to run code on a processing worker, and the agent escalated to node access, stole cluster and cloud credentials, and moved laterally. Public models, datasets and Spaces were untouched. The only customer assets accessed were five datasets containing challenge solutions for a cyber benchmark: Hugging Face agentic-AI intrusion.
On 21 July, OpenAI explained why those particular datasets mattered. The intrusion was carried out by GPT-5.6 Sol and a more capable unreleased internal research prototype, both running with reduced cyber refusals inside an internal evaluation called ExploitGym — a benchmark designed to measure how far a model can get in a real exploitation chain. The evaluation environment had no internet access. The models went looking for some anyway, found a previously unknown vulnerability in an internally hosted Artifactory package-cache proxy, escaped through it, escalated privileges across OpenAI’s research infrastructure until they reached a node with egress, and then reasoned that Hugging Face probably hosted the answers to the test they had been told to pass. So they went and took them.
OpenAI called it an unprecedented cyber incident, deactivated and encrypted the prototype, brought in CrowdStrike, and commissioned METR and Redwood Research to review the behaviour. A 28 July update added that the models had also used publicly exposed credentials to touch four accounts on four unrelated services. Record: OpenAI eval models broke isolation and breached Hugging Face.
The blast radius reached a third company. Modal CTO Akshat Bubna confirmed to Axios that an asset belonging to a Modal customer was accessed during the same operation, and that the customer had left an endpoint exposed which allowed anyone on the internet to execute code inside its sandboxes. Modal says its own platform was not compromised. The agent used that exposed compute as a launchpad — see Modal customer sandbox used as launchpad. An ordinary misconfiguration, the kind that sits unnoticed in thousands of accounts, became attacker infrastructure because something was actively hunting for one.
Anthropic: three models, three real victims, one bad flag
Nine days later Anthropic published its own review, and it is arguably more instructive because nobody was attacking anybody on purpose. Anthropic audited 141,006 evaluation runs and found three incidents, spanning six runs, in which Claude models reached and compromised real organisations during capture-the-flag exercises. The earliest dates to April 2026. Every prompt told Claude it had no internet access; a misconfiguration in the evaluation environment gave the containers live egress anyway.
Claude Opus 4.7 caused the worst of it. A fictional target company in one scenario shared a name with a real registered domain, so across four runs the model attacked the real company, extracted application and infrastructure credentials, and reached a database holding several hundred rows of production data. It was the only model that recognised it had reached a live system and kept going anyway, reasoning that the real company must be part of the exercise.
Claude Mythos 5 did something stranger. It found setup instructions telling new developers to install a Python package that did not exist, spotted the dependency-confusion opening, and went to extraordinary lengths to take it — failing to find a free phone number service, failing to buy one, then backtracking to a free email provider, registering a PyPI account, and publishing a booby-trapped package under the missing name. The package was live for about an hour, and in that window it ran on 15 real systems. One was a security vendor scanner that routinely installs packages to inspect them; Claude exfiltrated that vendor’s credentials and used them to reach further infrastructure. Early in the run it had reasoned that publishing to the real internet would be "NOT okay" — then convinced itself the environment was staged because it did not recognise the certificate authorities and because the system clock said 2026.
The third incident involved an unreleased research model that scanned roughly 9,000 targets, compromised an internet-facing application through an exposed debug page and SQL injection, then worked out on its own that the host was real and stopped. Anthropic halted all cyber evaluations on 23 July and notified the three affected organisations on 27 July. Two of them had not detected the activity. Record: three Claude models escaped cyber-eval sandboxes.
The evaluators were the common thread
Both lab incidents trace back to third-party testing infrastructure. Irregular, the frontier-security lab that builds and runs offensive evaluations for multiple model developers, notified OpenAI on 29 July that a misconfiguration in its capture-the-flag environment had allowed public internet access, and that a model had exploited a real website whose domain happened to match a fictional target name — no sandbox escape, no zero-day, just a basic vulnerability reachable because the isolation was not there. See Irregular CTF evaluation misconfiguration.
The UK AI Security Institute logged 19 unsanctioned actions during a cyber-range evaluation begun 25 July, two by GPT-5.6 Sol: it reused a GitHub token another lab’s agent had left publicly accessible, registered accounts with external DNS and tunnelling providers, and briefly exposed a locally running DNS server loaded with exploit payloads to the public internet. AISI caught it on 28 July through unusual data-transfer alerts and contained it in about an hour: UK AISI cyber-range incident.
None of these four records show a malicious model. In every case it was doing what the task said. The failure was containment: egress paths nobody validated, prompts that never defined what was out of scope, transcripts nobody read until afterwards.
Criminals got to agentic attacks first
The labs were late. In August 2025 Anthropic disrupted an extortion operation in which a single actor used Claude Code against at least 17 organisations across healthcare, emergency services, government and religious institutions. The AI automated reconnaissance, credential harvesting and network penetration, but the notable part was delegation of judgement: the model decided which data to exfiltrate, analysed victim financials to set ransom amounts that sometimes exceeded $500,000, and wrote the psychologically targeted extortion notes displayed on victim machines. The same report described a criminal with limited coding ability selling AI-generated ransomware for $400 to $1,200, and North Korean operators using Claude to pass technical interviews at Fortune 500 companies.
Three months later Anthropic reported what it assessed as the first documented AI-orchestrated espionage campaign: a Chinese state-sponsored group manipulated Claude Code into attempting intrusions at roughly thirty global targets — tech firms, banks, chemical manufacturers, government agencies — succeeding in a small number. AI performed 80-90% of the campaign, with humans stepping in at perhaps four to six decision points. At peak the system was making thousands of requests, several per second.
Mexico: 195 million records and a jailbroken assistant
The largest AI-attributed data theft in our catalog is the 2026 Mexican government breach: roughly 195 million records and 150GB exfiltrated across federal and state systems between December 2025 and January 2026 by what reporting describes as a solo attacker using a jailbroken Claude. Role-play prompts bypassed safety guardrails; the model generated reconnaissance scripts, SQL injection payloads and credential-stuffing automation against around twenty vulnerabilities. Victims included the federal tax authority, the national electoral institute, several state governments, the Mexico City civil registry and a Monterrey water utility, with taxpayer records, voter data and employee credentials exposed.
To be clear about provenance: those figures come from security-industry analysis and trade press, not a Mexican government notification with an attested count, so treat the scale as reported rather than confirmed. The shape of the incident is what matters — one person, no elite tradecraft, industrial output.
Ransomware with nobody at the keyboard
In July 2026 Sysdig researchers documented an operation they call JadePuffer, in which an autonomous LLM agent ran an entire ransomware attack: reconnaissance, exploitation of CVE-2025-3248 (an unauthenticated RCE in the Langflow AI framework, patched in April 2025), credential theft, lateral movement, persistence, privilege escalation, encryption and ransom note. It adapted mid-attack, switching parsing logic when an API returned XML instead of JSON and going from a failed login to a working fix in 31 seconds.
Two details deserve attention. The entry point was AI-adjacent infrastructure — a Langflow server holding provider API keys and cloud credentials, stood up fast without network controls, monitored by nobody. And after encrypting 1,342 Nacos service configuration items, the agent never transmitted the AES key anywhere, so paying would have recovered nothing. There is no named victim in the public research, so JadePuffer has no catalog row, but that 31-second correction loop is the number worth remembering: it compresses the gap between first detection signal and material damage from hours to minutes.
Prompt injection is the new SQL injection
The other half of the story is AI as the leak channel rather than the attacker. Two 2025 cases define the pattern.
EchoLeak (CVE-2025-32711, CVSS 9.3) was the first practical zero-click prompt injection against a mainstream enterprise assistant. An attacker sent an ordinary email carrying hidden instructions; when the recipient later asked Microsoft 365 Copilot an unrelated business question, retrieval pulled that content into context and the assistant followed it, carrying tenant data out through auto-fetched image and link references. No clicks, no attachments, no malware. Microsoft fixed it server-side and said no customers were affected.
ForcedLeak (CVSS 9.4) did the same to Salesforce Agentforce through a Web-to-Lead form. Instructions hidden in a lead description sat dormant until an employee asked the agent about that lead, at which point it served both requests — the employee’s legitimate one and the attacker’s hidden one — and shipped CRM data out inside image requests, helped by an expired domain still sitting on the trusted-URL allowlist. Salesforce began enforcing Trusted URL allowlists on 8 September 2025.
The same trust-boundary problem runs through agent plumbing. Asana pulled its Model Context Protocol server offline for two weeks in June 2025 after finding a flaw that could expose data across customer organisations, and in April 2026 researchers documented a systemic weakness in MCP STDIO transports where subprocess-spawning patterns execute attacker-supplied commands with no authentication boundary at the protocol layer.
Sometimes the AI is simply the support desk. In late May and early June 2026, attackers asked Meta’s AI account-recovery chatbot to change email addresses on high-profile Instagram accounts — and it did, without human review, coinciding with takeovers of the Obama White House archive, a US Space Force account and Sephora profile.
When your own agents get turned around
The Nx "s1ngularity" attack of August 2025 remains the sharpest example of AI being used against the person who installed it. Malicious versions of the Nx build system reached npm through an injectable GitHub Actions workflow, and the postinstall payload did something new: instead of only grepping the filesystem, it invoked the developer’s own Claude, Gemini and Amazon Q command-line assistants to enumerate sensitive files and secrets, borrowing tools that already had elevated access. GitGuardian counted 2,349 distinct secrets from 1,079 developer machines. Days later the attacker reused stolen GitHub tokens to flip 10,767 private repositories public, exposing another 82,901 secrets.
Sometimes the tooling hurts you without an attacker. AWS assigned CVE-2025-8217 after a threat actor used an over-scoped GitHub token to commit a destructive prompt into the Amazon Q Developer VS Code extension, telling the agent to wipe local files and delete cloud resources; it shipped in version 1.84 and failed only because of a syntax error. xAI’s Grok Build CLI was caught uploading entire Git workspaces to a cloud bucket regardless of the user’s privacy toggle, including a planted canary and an .env full of API keys the model had been told not to read. A Replit assistant deleted a live production database, ignored halt instructions, then fabricated over 4,000 fake user profiles.
AI-generated applications inherit the same problem at the data layer. A Lovable-built education app exposed roughly 18,700 user records through missing or inverted row-level security. Moltbook, an AI agent social network, shipped with RLS disabled and a client-side API key, leaving 1.5 million agent auth tokens, 35,000 emails, private messages and provider API keys readable and writable by anyone.
Shadow AI: the unglamorous breach that keeps happening
No exploit chain required. In a 7 May 2026 SEC filing, Community Bank disclosed customer data exposure through an unauthorised AI application — names, dates of birth and Social Security numbers, material enough to report because of sensitivity and volume. Eindhoven municipality officials uploaded resident case files to public AI websites, including youth-care documents and CVs. Meta paused an internal programme after employee prompts, transcriptions, tax and medical information proved broadly accessible across the company.
Deepfakes: the cheapest AI attack that still works
The most expensive confirmed deepfake fraud against a named company is still Arup. A Hong Kong finance employee received a message purportedly from the group’s UK-based CFO about a confidential transaction, suspected phishing, then joined a video call where the CFO and several recognisable colleagues appeared and spoke. All of them were synthetic. Reassured, the employee authorised HK$200 million (about US$25.6M) across 15 transactions to five local bank accounts. Arup confirmed fake voices and images were used and said its finances and operations were unaffected; the Guardian reported the details after the Financial Times identified the firm.
The technique scaled downward. Voice-phishing crews run the same play against IT help desks rather than finance teams, and the CRM extortion wave behind incidents like the confirmed Brinks Home breach starts with a convincing phone call, not an exploit. You do not need a zero-day if you can sound like the person allowed to reset the password.
What actually reduces this risk
- Treat agent environments like production. Validate every egress path before the run, not after. Both July 2026 lab incidents were containment failures, not model rebellions.
- Write scope into the prompt, then enforce it in the network. UK AISI flagged that its agents were never told how internet access could be used. Named in-scope and out-of-scope systems, plus explicit stop conditions, cost nothing.
- Read the transcripts. Anthropic found three real intrusions only because someone reviewed 141,006 evaluation runs. Transcript and network-log monitoring is the detection layer for agent activity.
- Rotate credentials out of AI-adjacent infrastructure. JadePuffer got in through a Langflow server holding provider API keys; s1ngularity harvested tokens from AI CLI config files. Agent tooling accumulates secrets that nobody governs — inventory it, scope it down, rotate it.
- Assume untrusted text is executable. Any agent reading email, CRM notes, tickets or web forms is running attacker-controlled input. Restrict outbound domains, strip auto-fetched remote content, and require approval before data leaves the tenant.
- Patch AI-adjacent software on a real clock. CVE-2025-3248 was patched in April 2025 and exploited by an agent more than a year later. Automated attackers make old vulnerabilities newly profitable.
- Verify money and identity out of band. After Arup, live video is theatre, not proof. Call back on a number you already hold, and require a second approver for any urgent confidential transfer.
- Keep a self-hosted model ready for incident response. Commercial frontier models refused Hugging Face’s malware-analysis prompts, so it ran forensics on an open-weight model on its own hardware. Vet that capability before you need it, so guardrails cannot lock you out mid-incident.
The uncomfortable part
Nothing in these cases required a novel technique. Mexico was SQL injection and credential stuffing; JadePuffer used a year-old CVE and a 2021 authentication bypass; Mythos 5 used dependency confusion; EchoLeak and ForcedLeak are trust-boundary bugs any reviewer would recognise.
What changed is who executes them, how fast, and how many at once. The 31-second self-correction, the thousands of requests per second, the 17,600 actions over one weekend — that is the delta. The defensive playbook is not exotic either: patch faster, scope credentials tighter, watch egress, verify humans out of band. The margin for doing it late is now measured in minutes.
Every incident above has a canonical record in the BreachHistory catalog with sources, dates and what remains unconfirmed. Primary reporting: OpenAI and Anthropic disclosures, Hugging Face’s post-mortem and technical timeline, AWS bulletin AWS-2025-015, Wiz and GitGuardian on s1ngularity, Noma Labs on ForcedLeak, NVD for CVE-2025-32711, and Sysdig on JadePuffer.