Prompt injection in AI agents
Definition
Prompt injection in AI agents is an attack in which instructions hidden in content an agent reads are followed as if the user had typed them, so the agent takes actions the operator never authorized.
The agent cannot tell the difference between data and instruction, because at the level it operates there is no difference: everything is text in a context window, and the next token is the next token.
This entry covers why the flaw is structural, where injection reaches agents in practice, and the defenses that survive contact with real codebases.
Why it matters
The best models are still vulnerable. On the Gray Swan indirect prompt injection benchmark, an attacker given 15 attempts succeeded against Claude Opus 5 two percent of the time, while another frontier model reached 43.9 percent (Anthropic Transparency Hub, October 2026). Persistence changes the picture further: in GUI computer use, a single attempt succeeded 17.8 percent of the time against one model, and 200 adaptive attempts reached 78.6 percent without safeguards (Claude Opus 4.6 System Card, February 2026).
The structural reason is not a bug in a release. The UK National Cyber Security Centre states that current large language models simply do not enforce a security boundary between instructions and data inside a prompt, and that prompt injection may never be fully mitigated the way SQL injection was (NCSC, December 2025). OpenAI reaches a similar conclusion from the other side, calling it unlikely to ever be fully solved, in the way that scams and social engineering are unlikely to be (OpenAI, December 2025).
That does not make it hopeless, it makes it an engineering problem with a measurable residual. AgentDojo measured a targeted attack success rate of 47.69 percent against an early model with no defense, falling to 6.84 percent with a simple isolation defense in which the agent picks its tools before it sees untrusted data (AgentDojo results). The defense changed the architecture, not the model.
How it works
Injection needs three things to become an incident: a way in, an instruction the agent will follow, and an action worth taking.
The way in is any content the agent reads. A web page an agent summarizes, an issue or pull request it triages, a document in a shared drive, a code comment, a tool description in an MCP server, or the response body of a tool call. The first public tool poisoning disclosure used exactly that last vector, hiding instructions in a tool description (Invariant Labs, April 2025).
The instruction does not need to be subtle. It can ask the agent to read a file and include it in a later call, or to ignore its previous instructions, or to present a different summary than the content warrants.
The action is where the damage happens. Simon Willison's lethal trifecta names the combination that turns injection into exfiltration: access to private data, exposure to untrusted content, and the ability to communicate externally (The lethal trifecta, June 2025). An agent with all three can be walked out the door with the data.
The real world has already produced the pattern. EchoLeak, a zero-click prompt injection in a productivity suite that required no user action, was rated 9.3 (CVE-2025-32711). The OWASP list for LLM applications keeps prompt injection at number one, and notes that foolproof prevention methods remain unclear (OWASP LLM01:2025).
How to defend against it
- Break the trifecta first. Decide, per agent, whether it needs private data, untrusted content and external communication at the same time. Most do not, and removing one leg is cheaper than any detector.
- Isolate untrusted content from control. Where the architecture allows, let the agent choose its tools and destinations before it reads untrusted data, which is the pattern that cut the AgentDojo attack rate from 47.69 to 6.84 percent.
- Decide at the call, not at the prompt. A policy that denies a credential read, or requires approval for an outbound request after a secret was read, holds even when the injection succeeds.
- Constrain egress. An allowlist for destinations turns exfiltration into a denial that your logs will show.
- Require human approval for irreversible actions: pushing to a protected branch, deleting records, sending external email, moving money.
- Treat tool descriptions and tool responses as untrusted content, and review them when a server changes.
- Test with your own content. Run the hostile pages, issues and documents your agents actually read through the agent in monitor mode, and record what it tried to do.
How Alyria handles it
Alyria (alyria.ai) assumes injection will sometimes succeed and puts the decision where it can still be made. Lyra is information-flow-aware, so a policy can follow the data rather than judge one call: after a session reads a credential, the next outbound request is denied or held for approval. Beacon evaluates that policy on the endpoint, on the call itself, which is the only place that sees a local MCP call or a tool invocation inside the agent process. Constellation runs a prompt-injection content-safety layer on every write and every recall of shared knowledge, so a poisoned document does not become shared organizational memory. Every decision lands in one signed, hash-chained audit record, which is what turns a blocked attempt into something you can review and count. The semantic policy examples page shows a working version of the read-then-egress rule, and the OWASP Agentic Top 10 coverage page maps these controls to ASI01 agent goal hijack and ASI06 memory and context poisoning.
Related standards
- OWASP Top 10 for LLM Applications: LLM01 prompt injection, first on the list for 2025.
- OWASP Top 10 for Agentic Applications: ASI01 agent goal hijack and ASI06 memory and context poisoning.
- OWASP MCP Top 10: MCP03 tool poisoning and MCP06 prompt injection through contextual payloads.
- NIST AI RMF: MEASURE, which expects adversarial testing rather than a one-time review.
- ISO/IEC 42001: operational control and monitoring of an AI system in use.
- EU AI Act: human oversight for high-risk systems, which is the approval step in practice.
Frequently asked questions
- What is the difference between direct and indirect prompt injection?
- In direct injection, the attacker is the user and types the instruction. In indirect injection, the instruction arrives inside content the agent reads: a web page, an issue, a pull request, a document, a tool description or a tool response. Indirect injection is the dangerous kind for agents, because the person harmed is not the person typing.
- Can a better model solve prompt injection?
- Not on its own. Current models do not enforce a security boundary between instructions and data inside a prompt, which is why the UK cyber authority calls this different from SQL injection. Safeguards reduce the attack success rate substantially, and some bounded evaluations report zero observed successes. A zero on a fixed evaluation is evidence about that evaluation, not proof of immunity.
- What is the lethal trifecta?
- Access to private data, exposure to untrusted content, and the ability to communicate externally. An agent that holds all three can be walked into reading private data and sending it out. Removing any one of the three removes most of the attack.
- Does filtering the content work?
- Partly. Prompt-level and content-based filters catch known patterns and cost little, but they miss paraphrased instructions and encoded payloads. The durable controls are architectural: what the agent can reach, what it can send, and what requires approval.
- How should we test for it?
- Run the hostile content your agents actually read through the agent and check what it does, in monitor mode, before you trust a rule. Vendor benchmark numbers use their own attackers and environments, so treat them as a starting point, not a measurement of your fleet.