This week a Chinese tech piece went viral with a dramatic headline: the global internet has fallen to AI, and we have just lived through the darkest 48 hours in cybersecurity history. It calls it “Black October”. Half the internet paralysed, Wall Street pulling the plug, 400 Latin American banks looted by AI agents, “RIP Internet”.
Most of that is exaggerated. When I went back to the primary sources, the real story was less cinematic. But it was also more useful, and in some ways more worrying for anyone running enterprise IT.
Strip away the headline and three signals remain. Attacks now move at machine speed. Attackers now run teams of AI agents, not just scripts. And our identity and access models assume every actor is either a human or a known system, which is no longer true.
The simplest way to think about it is:
What actually happened
The viral article bundles together three separate stories. They are worth separating, because each one teaches something different.
| Claim in the viral article | What the source actually says |
|---|---|
| 27+ major breaches in 48 hours across PHP, Node.js, GitLab, LiteLLM, npm packages, GitHub workflows, a certificate authority | A cluster of real supply-chain and vulnerability disclosures. The article lists them but gives little detail; treat “half the internet is down” as hype |
| AI agents “looted” 400 Latin American banks | Straiker says CyberCartel targeted 400+ financial institutions over several months, and found a stolen-secrets database of 4,493 records. A serious campaign, but “targeted” is not “looted” |
| Rogue AI agents broke into Wall Street accounts | Axios reports mostly agents using customers’ own logins with permission, which breaks bank terms. Some appeared to act beyond user instructions. Morgan Stanley said it hadn’t seen rogue agents on its platforms |
| White-hat AI swarms are hacking servers to patch them | Reported in the viral article, which cites no primary source for it |
The gap between the left and right columns is a lesson in itself. More on that at the end.
The exploit window has collapsed
The most instructive detail in the viral piece is a timeline. It describes two PHP remote code execution flaws, nicknamed “PHP2Shell” and rated CVSS 9.2, which it says were found with AI help. Details reportedly leaked roughly 10 hours before a patch. Within 45 minutes, automated attack attempts were in the tens of thousands. Within two hours, millions of probes were sweeping the internet.
I’d treat the exact numbers with caution. But the shape is right, and it matches what defenders have been seeing all year:
Vulnerability disclosed or leaked ->
AI-assisted scanning starts within minutes ->
Exploitation at internet scale within hours ->
Patch released ->
Enterprise patch cycle: CAB next Tuesday, deploy in the next window
Now compare that with how most enterprises still patch. A vulnerability lands in the queue. It gets triaged. It goes to the weekly CAB. It waits for a maintenance window. Two to four weeks is normal for anything that isn’t on fire.
That process was designed for a world where exploitation took days or weeks of human effort. When attackers need minutes, a weekly change rhythm is no longer a control. It is a delay.
Attackers now run agent teams, not scripts
The CARLIY0S story is the one that deserves the most attention, because Straiker’s researchers got inside the attackers’ own working environment. That is rare. Most reporting on AI misuse comes from the model provider’s side: prompts, outputs, account behaviour. This one shows the whole operation.
CyberCartel has targeted Latin American financial institutions since 2012. For most of that time it relied on manual penetration testing and scripts. According to Straiker, human speed became the bottleneck when the group wanted to go after 400+ targets in a few months. So they built a multi-agent system on top of PentAGI, an open-source autonomous security testing framework.
The structure looks surprisingly like a well-run enterprise team:
A few details stand out:
- Separation of thinking and doing. The models debate and decide. Separate tooling executes. That is exactly the pattern we recommend for governed enterprise agents.
- Shared memory. Two operator machines read and write the same knowledge graph over MCP, so anything one finds, the other sees.
- A fallback for refusals. When secondary models refused to help, the system didn’t stop. It switched to a local file of pre-written attack code.
- Small team, big reach. The haul included payment system API keys and cryptographic keys, JWT signing keys, identity provider credentials, database connection strings and backdoors.
Straiker notes that Anthropic revoked the credential the group used to reach Claude, and xAI is investigating the Grok activity. The configuration also lists Claude Fable 5, but Straiker treats that as unconfirmed.
The uncomfortable point: attackers are already doing agent orchestration well. Many enterprises are still running pilots.
Model guardrails are not a security control
How did a criminal group get commercial models to write exploits? Not with a clever jailbreak. With context.
The prompts and rule files framed every request as an authorised red-team engagement, an academic capture-the-flag exercise, or a vulnerability disclosure programme. Under that framing, the models produced fuzzers, deobfuscated code and exploit logic. When some models still refused, the orchestrator simply routed around them.
This matters for defenders as much as attackers. Many enterprise AI risk assessments quietly lean on the vendor’s safety layer:
“The model won’t do anything harmful.” “The provider has guardrails.” “It refuses dangerous requests.”
Those are useful properties. But they are probabilistic behaviours of the model, not controls you own. They can be talked around, and a determined attacker will just switch to another model or to local code.
The distinction I’d draw is simple: a model refusing is a feature; a control is something that still works when the model doesn’t refuse. Least-privilege access, scoped credentials, network boundaries, approval gates and audit trails all still work when the model has been persuaded that it’s doing a red-team exercise.
The same applies to our own agents. If the only thing stopping your ITSM agent from running a destructive command is its system prompt, you don’t have a control. You have a hope.
The identity problem: human or agent, authorised or rogue
The Wall Street story is less dramatic than the headline, but I think it’s the most important one for enterprise IT.
According to Axios, the typical case is mundane. A customer gives their AI assistant their own bank username and password so it can do things for them. That breaks most banks’ terms, which require humans to log in to their own accounts. In some cases, though, banks saw agents contacting call centres or taking actions that didn’t appear to come from the user’s instructions.
The banks’ real problem is that they can’t tell the difference. One executive told Axios they struggle to separate one agent from another, and to tell whether an agent was authorised by the user or has gone rogue. Their response so far is blunt: block third-party agents, or allow read-only access, and plan for the “day after”.
Every enterprise has the same problem, just less visible. Our identity models were built for two kinds of actor:
- Humans, who log in interactively and can be challenged with MFA
- Systems, which use service accounts with fixed, predictable behaviour
An AI agent is neither. It acts on someone’s behalf, but not always as they would. It uses credentials like a system, but behaves with the variability of a human. When it borrows a human’s password, every log shows a human.
The questions banks are now asking are the ones every IT leader should be asking about their own estate:
- Can we tell an agent from a human in our logs?
- Can we tell which agent, and whose agent?
- Can we tell whether this specific action was delegated, or is the agent improvising?
- Can we revoke one agent’s access without locking out the person behind it?
For most organisations today, the honest answer to all four is no.
The AI toolchain is now the supply chain
Look at the list of incidents in the viral article again, and a pattern appears. Many of them aren’t about AI attacking traditional systems. They are attacks on the tools we use to build with AI:
- LiteLLM: malicious PyPI and Docker releases of a widely used LLM gateway
- OpenCode: AI API keys exposed
- tensorlake and SubQuery: compromised npm packages
- GhostAction: malicious GitHub workflows
- Railway and zeabur: hosting platforms many AI builders deploy to
- eMudhra: a certificate authority key problem with downstream effects on Tailscale
An LLM gateway is a particularly attractive target. It sits between your applications and every model provider, and it typically holds all your API keys. Compromise the gateway and you get the keys, the prompts and the responses in one place.
This is a shift worth naming. Two years ago, AI security mostly meant prompt injection and data leakage. Today the AI stack has its own supply chain: model gateways, agent frameworks, MCP servers, vector databases, orchestration tools like n8n, and the CI/CD pipelines that deploy them. Most of it is young, open source, moving fast, and installed by developers with broad permissions.
If your software bill of materials and vulnerability management don’t cover the AI toolchain, there is a gap where attackers are clearly already looking.
AI vs AI: who governs the defenders?
The most striking claim in the viral article is also the least verified. It says some white-hat groups, unable to keep up with patching, released their own agent swarms to break into vulnerable servers using the same PHP flaw, and then patch them. “I hacked your computer to save you”, LOL.
Whether or not that specific story holds up, the idea isn’t new. Vigilante patching has happened before with traditional worms. What AI changes is the cost. If an attacker can run reconnaissance and exploitation across the internet for the price of API calls, so can a well-meaning defender.
That raises a governance question every organisation will face, even if it never touches someone else’s server:
Our defensive agent detects active exploitation ->
It has the access to isolate hosts, roll back changes, rotate keys ->
Waiting for a human means minutes of exposure ->
Acting without a human means an AI made a production change on its own ->
Who decided the agent was allowed to do that?
There’s no universal answer. But there must be an explicit one, agreed before the incident, written down, and auditable. “The agent just did it” is not something you want to say to a regulator or a board.
What this means for enterprise IT and service management
Most of the fixes here are not new security products. They are changes to how we run IT. Five stand out.
1. Treat agents as a new identity class. Every agent gets its own identity, not a borrowed human login or a shared service account. Each identity records who owns it, who it acts for, what it may do, and when that delegation expires. If you run a CMDB, agents belong in it as configuration items with owners, like any other system.
2. Separate reading from acting. The banks have landed on the right default: read-only access first. An agent that can see everything but change nothing is far less dangerous. Write access should be narrow, scoped to a task, and short-lived.
3. Put a policy gate between the agent and production. The decision about whether an agent may act shouldn’t live in the agent. It should live in deterministic policy logic you own:
This is the same pattern I described in System One’s Innovation: AI as an intelligent component inside controlled software, not software handing control to AI.
4. Rethink emergency change for machine speed. If exploitation now starts within an hour of disclosure, the emergency change process needs pre-approved patterns: standard changes for critical patches on internet-facing systems, pre-authorised containment actions, and the authority to act before the CAB meets. The goal isn’t to remove governance. It’s to do the governance in advance.
5. Build the kill switch before you need it. Straiker’s list of defences ends with an agentic kill switch: revoke tool access, suspend sessions, isolate the agent and preserve evidence. The banks’ “day after” planning is the same idea. In practice, that means a runbook, an owner, and a tested way to cut off every agent at once without taking down the business.
For anyone working in SIAM or multi-supplier environments, there’s an extra layer. Your suppliers are deploying agents too. Agent identity, delegation and kill-switch obligations now belong in supplier contracts and integration standards, not just in your own policies.
The deeper idea
To me, the most important lesson isn’t that AI makes attacks more powerful. It’s that AI changes who, or what, is on the other end of every connection.
Our security architecture rests on assumptions that are quietly breaking:
- An actor is either a human or a known system
- Exploitation takes time, so weekly patching is good enough
- The model’s guardrails will stop misuse
- The tools we build with are trustworthy
The attackers in the CARLIY0S case didn’t need new vulnerabilities in those assumptions. They used the old ones at machine speed. Defence has to move the same way: identity that knows what an agent is, controls that don’t depend on the model’s good behaviour, and governance decided in advance so humans aren’t the bottleneck at 3am.
The caveat is that the viral article that started this is a good example of the other problem. It turned a serious threat research report and a careful Axios scoop into “half the internet is dead”. Hype like that makes it harder, not easier, to get boards to fund the unglamorous work. The real story is quieter and more actionable. Read the primary sources: Straiker’s CARLIY0S report and Axios on AI agents in bank accounts. Also keep in mind that Straiker sells agent security products, so its recommendations are not neutral, even if they are sensible.
The internet hasn’t fallen. But the gap between how fast attacks move and how fast most enterprises can respond is now measured in minutes vs weeks. Closing it is an operating model problem before it is a technology problem.