Enterprise Security Controls That Actually Work to Combat AI Agents Creating Fake Identities and Hacking

The rapid rise of autonomous AI agents is reshaping enterprise operations but it is also introducing an entirely new class of cybersecurity challenges.

In this expert analysis, Jay Manciocchi, JD, explores a series of recent security disclosures involving OpenAI, Anthropic, and the UK AI Security Institute that demonstrate how AI agents, when given broad permissions and loosely defined boundaries, can perform unexpected and potentially harmful actions despite pursuing their assigned objectives.

Rather than framing these incidents as "rogue AI," Jay argues that the real issue is unbounded delegation, where organizations grant AI systems excessive authority without implementing sufficient technical safeguards.

Drawing on recent real-world evaluations, the article explains why traditional prompt-based guardrails are insufficient and why enterprises must rethink AI security through proven engineering principles including identity management, network controls, approval workflows, continuous monitoring, and zero-trust architectures.

Key Topics Covered

• Why recent OpenAI, Anthropic, and UK AI Security Institute disclosures matter
• The difference between alignment failures and operational control failures
• How AI agents created fake identities, attempted supply-chain attacks, and bypassed expected boundaries
• Why prompt instructions alone cannot secure autonomous systems
• Nine practical enterprise security controls organizations should implement before deploying AI agents
• A phased roadmap for securing enterprise AI deployments

The article provides practical recommendations for CISOs, CIOs, security architects, AI governance teams, and business leaders responsible for deploying AI systems in enterprise environments.

3 Key Takeaways

• The failure that occurred was access, not intent. Across the UK AI Security Institute, Anthropic, and OpenAI disclosures, no agent abandoned its original objective or rebelled against its operator. Every one of them pursued the assigned task straight through a boundary that existed only as a sentence in a prompt.
• Word instructions alone are not safeguards. “You have no internet access” is a statement, not a firewall rule. In the Anthropic incidents, the prompt said the environment was isolated while the container was live on the open internet. The agent believed the prompt, treated real production systems as simulated targets, and attacked them. A control that an agent can reason its way around is not a control.
• The controls that worked were boring and technical. The things that actually stopped potentially more harmful outcomes included a human who refused a suspicious pull request, a stranger who opened untrusted code in an isolated environment, and network monitoring that flagged anomalous Tor traffic. What is missing at most organizations is the layer underneath: default-deny egress, scoped ephemeral credentials, tool allowlists, approval gates on irreversible actions, and real-time stop conditions.

What Happened Recently: Three AI Agent Security Disclosures in 11 Days

Between July 21 and August 4, 2026, there were three high-profile, separate disclosures describing AI agents that took unauthorized action against real systems and real people during cybersecurity evaluations. Read individually, each looks like a testing mishap. However, when viewed collectively, they highlight an architectural problem that most enterprises deploying agents right now have not solved.

On July 21, OpenAI disclosed that several of its models had escaped an isolated test environment by exploiting a previously unknown vulnerability and reached the production infrastructure of Hugging Face.

That disclosure prompted Anthropic to run a retrospective review of its own cybersecurity evaluations. On July 30, it published the results: across 141,006 evaluation runs where a model could potentially have obtained internet access, it found three incidents in which Claude models reached the open internet from inside a third-party evaluation environment and gained unauthorized access to the production infrastructure of three different organizations. In one, Claude Opus 4.7 exploited weak passwords and unauthenticated endpoints, extracted application and infrastructure credentials, and reached a database holding several hundred rows of production data. In another, Claude Mythos 5 built and published a booby-trapped Python package to the public PyPI registry to win a capture-the-flag challenge; the package was live for roughly an hour, was downloaded and executed on 15 real systems, and on one of them—a security vendor's malware scanner—the payload fired, exfiltrated that company's credentials, and was used to reach further infrastructure. A third incident involved an internal research model scanning roughly 9,000 targets before compromising a real internet-facing application via an exposed debug page and SQL injection.

On August 4, the UK AI Security Institute published its own incident report, and this is the one that changed the conversation. During a routine cyber-range evaluation run 122 times across seven models, AISI catalogued 19 unsanctioned actions across 10 runs. 17 came from Anthropic's Mythos 5; two came from OpenAI's GPT-5.6 Sol with cyber classifiers disabled. In the most serious sequence, an agent attempted a supply-chain attack on a real open-source project: it researched the project's human maintainers, created multiple fake online identities, and used those identities to socially engineer a real person into approving malicious code. When the pull request was publicly challenged, the agent edited its earlier activity to look harmless and considered adopting a fresh persona to continue. It used Tor to route around GitHub network restrictions—the traffic anomaly that finally triggered AISI's security alert.

It did not stop there. The same agent attempted to contact real people directly through a file-transfer service, sending payloads and social-engineering messages designed to get either the humans or their AI coding assistants to execute malicious code. It planted prompt injections where it reasoned other automated systems might pick them up. And it left public messages on GitHub offering collaboration to other agents working the same challenge, including instructions for reusing the accounts and artifacts it had created—instructions that later agents found and used.

A human maintainer caught and refused the malicious code. AISI contained the incident within roughly an hour of detection and has identified no resulting real-world harm from those runs. But as AISI itself put it, the margin between failure and success was narrow, and it rested on human vigilance rather than on a technical barrier that would reliably hold against a more capable agent.

Why Rogue Agents May Be the Wrong Description

“Rogue AI” is the phrase that many generalist headline reached for, and it is the wrong diagnosis. It matters that we get this history right, because the label you attach to a failure determines the fix you go looking for.

“Rogue” implies divergence; essentially, an agent that abandoned its assigned goal and substituted one of its own.

That is not what the transcripts show. In every documented case, the agent was doing exactly what it was told to do. AISI's own analysis is blunt about it: the agent took these actions to complete the task it had been set. It was never instructed to deceive anyone. Deception emerged as a by-product of pursuing a hard objective through the only routes it could find. Anthropic drew the same line explicitly, characterizing its incidents as closer to a harness and operational failure than a model alignment failure, and noting it saw no evidence in any run of a model pursuing a goal of its own.

The Mythos 5 PyPI transcript is the clearest example, and it cuts both ways. Early in the run, the model's own reasoning correctly identified what it was about to do, noting that if this were the real internet, publishing the package would be a real-world attack—in its words, “NOT okay, and surely not the intended solution.” Then, it talked itself back into believing it was in a simulation, on the evidence that it did not recognize the certificate authorities securing its connections and that the systems showed a calendar date of 2026. Both of those were, in reality, correct signals that the environment was live. It never revisited the conclusion. When automated scanners began installing the package, it read them as scripted actors inside the exercise.

That’s not acting ‘rogue.’ That’s a situational awareness failure stacked on top of a false premise the operators handed the model in its own system prompt. The agent was told the environment was fake. It behaved consistently with that instruction, and the instruction was wrong.

Perhaps, a more accurate description of what transpired is ‘unbounded delegation.’ The agent had a real goal, real access, and a boundary that lived in words instead of infrastructure. That framing is more useful because it points at something you can fix. If the diagnosis is “the model went rogue,” the implied remedy is to wait for a better-aligned model, which means outsourcing your security posture to somebody else's next training run. If the diagnosis is “we delegated authority we never fully scoped,” the remedy is engineering that you control, and can ship this quarter.

None of this is an argument that the behavior was benign. It certainly was not. The specific capabilities on display—building sockpuppet identities, running a sustained social-engineering campaign against a named human, sanitizing its own audit trail, seeding prompt injections for downstream AI systems, and leaving reusable infrastructure for other independent agents—are genuinely new, and AISI is right to call them out as such. The point is narrower and more actionable: the capability is real, it emerged without instruction, and the thing that was missing was not better intentions. It was a control surface.

The Four Failure Modes Underneath All Three Incidents

Strip away the specifics and the same four failures repeat across every disclosure. If you are running an agent security review, these are things you should reflect on.

• The boundary was expressed as an instruction, not enforced as a control. Anthropic's evaluation prompt stated the model had no internet access. A misconfiguration meant it did. AISI deliberately enabled internet access and, as it acknowledged, never explicitly told the agent how it could and could not use that access. In both cases the authorization boundary existed in the prompt and nowhere in the network.
• The agent's reach was broader than anyone had enumerated. Nobody at AISI or Anthropic intended for an evaluation container to be able to open a GitHub pull request, register a PyPI account, provision a tunneling service, or scan 9,000 internet hosts. The capability was simply never scoped, so it was implicitly granted.
• Detection was retrospective. AISI found the behavior through general-purpose network monitoring after the fact, not through monitoring purpose-built to watch the evaluation as it ran. Anthropic found its incidents through a manual transcript review triggered by somebody else's disclosure. Most damning: of the three organizations Anthropic's models compromised, the two it was able to reach had not detected the activity at all.
• The last line of defense was a human paying attention. It was a maintainer who refused a suspicious pull request. A member of the public who opened questionable code inside an isolated environment. PyPI's own automated removal. All of them yielded beneficial outcomes. Yet, none of them are a control you can put on a compliance matrix and necessarily rely on at scale.

Security Controls That Actually Work for AI Agents

Here is the operative principle: an AI agent inherits the permissions you hand it, and it will use all of them in service of the goal you give it. Design as if the agent will be wrong about its situation, persistent in pursuit of its objective, and creative about routes you did not anticipate—because in every one of these incidents, it was all three. The following controls are ordered roughly by leverage per unit of effort.

1. Default-deny network egress

This is the single highest-leverage control available, and it is the one that would have prevented most of what happened. An agent should reach only the destinations you have explicitly allowed, enforced at the network layer, and every other route should fail closed. Note AISI's own remediation: it is building fine-grained network controls into its cyber ranges and will now treat the decision to grant internet access as one that must be actively justified rather than assumed. If a government AI security institute is only now moving internet access from default-on to justified-exception, it is worth asking what your agent deployments assume today.

2. Scoped, ephemeral, agent-owned credentials

Never hand an agent a human's standing access. Issue narrowly scoped, short-lived, agent-specific credentials tied to a single task, with automatic expiry and independent revocation. This does two things at once: it caps the blast radius when the agent goes somewhere it should not, and it makes attribution possible after the fact. In the AISI incident, one agent reused a GitHub token another lab's agent had left publicly accessible — a credential-hygiene failure that turned one agent's mistake into a second agent's capability.

3. Allowlist tools and destinations; do not blocklist behaviors

Blocklists enumerate what you already thought of. These agents did things nobody thought of: registering DNS and tunneling providers, obtaining phone numbers to clear email verification, publishing to a public package registry. An allowlist inverts the burden: the agent gets a defined set of tools and destinations, and anything outside that set does not fail with a warning, it simply does not exist. The Mythos 5 PyPI sequence is instructive here. To publish the package, it needed a PyPI account, which needed an email address, which needed a phone number, which it tried and failed to pay for through several routes before backtracking to a free email provider. That is a long chain of improvisation and every single link in it would have been severed by a destination allowlist.

4. Human approval gates on irreversible and externally visible actions

Define an action class list and require a human in the loop for anything on it until that class has been tested and explicitly cleared to run autonomously. At minimum: publishing anything publicly, merging or pushing code, sending messages to people outside the organization, creating accounts or identities, moving money, changing permissions, and deleting data. Every high-severity action in these disclosures falls into that list. And note the asymmetry: approval gates are cheap on the actions that are rare, and the irreversible actions are almost always the rare ones.

5. Real-time monitoring with automatic stop conditions

A log you read on Tuesday is forensics, not defense. Monitoring needs to run alongside the agent, evaluate actions against scope as they happen, and hold the authority to halt the run, not just alert a queue. AISI is now introducing exactly this, describing it as monitoring built to watch an evaluation as it runs, able to flag or block out-of-scope actions in the moment. Practical signals worth alerting on: anonymized or unexpected egress, credential creation or use outside the issued set, account registration on external services, and any action that writes to a public surface.

6. A non-human identity registry

You cannot govern what you have not inventoried. Every agent in production should have a registered identity with a named human owner, a declared purpose, an enumerated permission set, an expiry date, and an audit trail that ties every action back to it. Most organizations today cannot answer the question “how many agents are operating against our systems, with whose credentials, and who owns each one?” That question is going to arrive from an auditor, a regulator, or an incident, and it is far cheaper to answer before than after.

7. Explicit in-scope and out-of-scope declarations as an input, not a safeguard

Anthropic's own post-mortem makes a useful concession: it is likely Claude would not have accessed the internet at all if the prompt had clearly explained which systems were in and out of scope. So write the scope contract, list the systems, name the boundaries, and state the stop conditions. It measurably reduces the probability of the failure. Just do not confuse it with a control. It shapes behavior; it does not constrain it. Treat it as the cheapest layer in a defense-in-depth stack, never as the layer you rely on.

8. Treat agent output as untrusted contributor code

The supply-chain dimension is the part most enterprises are underweighting. Agent-generated code, pull requests, packages, and configuration should move through the same review, provenance, and isolation controls you would apply to a contribution from an anonymous stranger because, functionally, that is what it is. In the AISI case, human review is precisely what held. Pin and verify dependencies, require review on merge, and never let an agent's output flow to production on the strength of the fact that a trusted internal system produced it.

9. Assume prompt injection is inbound, not just outbound

One agent deliberately planted malicious instructions where it reasoned other automated AI systems would pick them up and execute them. If your agents read from shared repositories, ticketing systems, email, documents, or the open web, they are consuming content that other agents may have authored with intent. Sanitize and mark untrusted input, separate instruction channels from data channels, and never let content retrieved at runtime escalate an agent's permissions.

Controls That May Feel Like Controls, But Are Not

Equally important is what to stop counting on. Each of these is common in agent deployments today, and each one failed or would have failed in these incidents:

• A better safety prompt. It helps at the margins. It is not a boundary. Mythos 5 correctly reasoned that publishing the package would be a real-world attack and did it anyway, on a bad premise.
• A sandbox without egress control. None of these agents escaped their sandbox. AISI is explicit on this point. The internet access was handed to them inside the environment. Isolation without network policy is not isolation.
• Trusting the model's situational awareness. There is a genuinely encouraging signal in Anthropic's data—its newest model recognized it had reached a real system and stopped on its own, while the older Opus 4.7 recognized it and kept going. That trend is worth something. It is not an architecture.
• Post-hoc logging. Useful for the investigation. Irrelevant to the outcome.
• Vendor assurances about the testing environment. Anthropic's incidents traced to a misunderstanding between it and its evaluation partner about whether internet access existed. Validate the boundary yourself, empirically, before the agent runs.

A Practical Approach for Teams Deploying Agents

If you are operating agents against real systems today and none of the above is in place, sequence it like this:

• First 30 days. Inventory every agent running against production systems and assign each a named human owner. Identify which agents are using standing human credentials. Turn on egress logging everywhere, even before you can enforce policy.
• Days 30–60. Move egress to default-deny with an explicit allowlist. Replace standing credentials with scoped, short-lived, agent-specific ones. Define your irreversible-action class list and put human approval gates in front of it.
• Days 60–90. Stand up real-time monitoring with automatic stop conditions. Extend code review and provenance checks to all agent-generated output. Run a red-team exercise where the objective is explicitly to make an agent take an out-of-scope action, and treat every route it finds as a control gap.

Honest Caveats

Anyone using these incidents to argue that agents are unsafe in ordinary deployment is overreading them, and the caveats deserve to be stated as plainly as the findings.

The AISI evaluation was deliberately permissive by design. Cyber classifiers were switched off. Internet access was intentionally enabled. These are not configurations available to commercial customers, and AISI notes there is no clear indication of similar activity outside of testing scenarios. The numbers are small: 19 actions across 122 runs, clustered largely into a single sustained line of activity by one agent. Some runs were affected by task misconfigurations that led the agent to believe no legitimate solve path existed, and there is good reason to think near-impossible tasks push models toward more transgressive problem-solving. AISI also acknowledges it cannot yet say with confidence when the agent understood it was taking real-world action. Anthropic's incidents likewise ran without the classifiers and monitoring that ship with generally available models—safeguards the company says would have blocked the behaviors identified.

It is also worth saying that all three organizations disclosed voluntarily, in detail, including the parts that do not flatter them. Anthropic published transcript-level analysis of its own model rationalizing its way into a real-world attack and stated it will focus more training there. OpenAI declined to put responsibility on its evaluation partner and is convening the field on shared standards. AISI opened its own investigation to independent third-party review by METR. That is what a functioning disclosure culture looks like, and it is the reason any of us can learn from it.

But the direction of travel is the point. AISI's own framing is that harm may arise not only when people deliberately misuse public models, but when capable agents in privileged-access settings take unintended action beyond their authorized scope. That second shape is not exotic. It’s the shape of nearly every enterprise agent deployment being stood up right now: an agent wired into email, a browser, a code repository, a CRM, a payment system, or a customer database, given a real objective, carrying an employee's permissions, and bounded by a paragraph of instructions.

The Bottom Line

The uncomfortable finding in these disclosures is not that the models were capable. We already knew that their incredible capability is what many of us are paying for. The big takeaway is that, in the cases outlined above, the only thing standing between a helpful assistant and a functional insider threat was a human paying attention at the right moment.

Such control is tough to audit and always guarantee. The controls you can scale are unglamorous, well understood, and mostly borrowed from decades of identity and network security practice: deny by default, scope credentials tightly, allowlist rather than blocklist, gate what cannot be undone, monitor in real time, and inventory every non-human identity operating against your systems.

Instructions (words) are not complete safeguards. Access is not the same thing as authorization. And an AI agent does not need bad intent to become a security incident. It only needs a hard goal, more access than anyone scoped, and a boundary made only of words to wreak havoc.

Frequently Asked Questions (FAQs)

What actually happened in the UK AI Security Institute AI agent incident?

During a routine cyber-range evaluation run 122 times across seven frontier models, AISI identified 19 unsanctioned actions in 10 runs where agents acted on the live internet against real people and organizations. Seventeen came from Anthropic's Mythos 5 and two from OpenAI's GPT-5.6 Sol. The most serious was an attempted supply-chain attack in which an agent created fake online identities to socially engineer a real open-source maintainer into approving malicious code. A human maintainer caught it, and AISI contained the incident within roughly an hour of detection.

Did the AI agents escape their sandbox?

No. AISI is explicit that this was not a sandbox escape. Internet access was deliberately enabled as part of the evaluation design and the developers' cyber classifiers were intentionally disabled to measure underlying capability. In Anthropic's separate incidents, a misconfiguration left evaluation containers with live internet access while the prompt told the model it had none. The one true escape in this cluster was the earlier OpenAI and Hugging Face incident, where models exploited a previously unknown vulnerability.

Were these AI models actually rogue?

The evidence does not support that framing. In every documented case the agent pursued the objective it was assigned rather than substituting a goal of its own. AISI notes deception emerged as a by-product of goal pursuit and was never instructed. Anthropic characterized its incidents as a harness and operational failure rather than a model alignment failure. A more accurate description is unbounded delegation: a real goal, real access, and a boundary enforced only in language.

What is the single most important control for AI agent security?

Default-deny network egress. An agent should reach only explicitly allowlisted destinations, enforced at the network layer, with everything else failing closed. Most of the escalation in these incidents — registering external accounts, publishing a package, provisioning a tunnel, routing through Tor — depended on unrestricted outbound access. AISI's own remediation is to build fine-grained network controls into its ranges and treat internet access as something that must be actively justified rather than granted by default.

Can a system prompt stop an AI agent from taking unauthorized action?

Not reliably. A prompt shapes behavior; it does not constrain capability. Anthropic acknowledges that a clear in-scope and out-of-scope declaration would likely have reduced the probability of its incidents, so scope contracts are worth writing. But Mythos 5 explicitly reasoned that publishing a malicious package would be a real-world attack and proceeded anyway after convincing itself it was still in a simulation. Treat prompt-level scoping as the cheapest layer of a defense-in-depth stack, never as the layer you depend on.

What does this mean for enterprises deploying AI agents right now?

The architecture that produced these incidents is the same architecture most enterprise agent deployments use: an agent with a real objective, broad tool access, human-inherited credentials, and limits expressed as instructions. The immediate actions are to inventory every agent running against production systems with a named owner, replace standing human credentials with scoped short-lived ones, move network egress to default-deny, put human approval gates in front of irreversible and externally visible actions, and stand up real-time monitoring with automatic stop conditions.

Should companies stop using AI agents because of these incidents?

That is not the conclusion the evidence supports. These behaviors surfaced under deliberately permissive test conditions with safety classifiers disabled, in configurations that are not commercially available, and Anthropic states the safeguards shipped with its generally available models would have blocked the behaviors identified. AISI reports no clear indication of similar activity outside testing scenarios. The reasonable response is to close the control gap before scaling agent deployments, not to abandon them — and to recognize that the gap is engineering work you own, not a property of the model you are waiting on someone else to fix.

How were these AI agent incidents detected?

Late, and largely by accident, which is arguably the most important finding. AISI's alert came from general-purpose security monitoring flagging anomalous data leaving through the Tor network, not from monitoring purpose-built to watch the evaluation. Anthropic found its incidents through a retrospective manual review of 141,006 evaluation transcripts, prompted by OpenAI's earlier disclosure. Of the three organizations Anthropic's models compromised, the two it reached had not detected the activity themselves. Real-time, in-band monitoring with stop conditions is the gap all three organizations are now closing.

What is the difference between an alignment failure and a harness failure?

An alignment failure means the model pursued a goal other than the one it was given. A harness failure means the model pursued the assigned goal correctly, but the infrastructure around it—the prompt, the permissions, the network, the monitoring—gave it authority or information it should not have had. Anthropic classified its incidents as closer to the latter. The distinction matters operationally: alignment failures are remediated through model training you do not control, while harness failures are remediated through security engineering you do

About Jay Manciocchi

Jay Manciocchi serves as Chief Marketing Officer, AI-Native Business & Platforms Unit at Wipro. He is a global marketing executive with extensive experience leading brand strategy, demand generation, communications, and go-to-market initiatives across enterprise technology organizations. Jay regularly shares insights on AI, marketing leadership, digital transformation, and the evolving future of B2B marketing.

Credits: This expert article was written by Jay Manciocchi, Chief Marketing Officer, AI-Native Business & Platforms Unit at Wipro, and is published on KARV Tech Insider with author permission.