What the research says about prompt injection and tool misuse in AI agents, and how to contain it

Large language models do not keep instructions and data apart. The UK National Cyber Security Centre says so directly: LLMs "simply do not enforce a security boundary between instructions and data inside a prompt," and prompt injection "may never be totally mitigated in the way that SQL injection attacks can be" (NCSC, December 8, 2025).

That one property decides how an agent programme has to be built. The question is not whether a model can be made to ignore an instruction hidden in a web page, a GitHub issue or a tool description. It cannot, reliably. The question is how often it fails, and what catches the failure before it reaches your data, your repository or production.

Both halves have published numbers. On the Gray Swan indirect-prompt-injection benchmark, an attacker given 15 attempts succeeded against Claude Opus 5 2.0% of the time, and GPT-5.6 Luna reached 43.9% (Anthropic Transparency Hub, October 2026). Tool calls go wrong in ordinary use too: the top overall entry on the Berkeley Function Calling Leaderboard scores 77.47% accuracy, and 68.38% on multi-turn function calling (BFCL, data last updated April 12, 2026).

The vendors do not expect that to close by itself. OpenAI's position is that prompt injection, "much like scams and social engineering on the web, is unlikely to ever be fully 'solved'" (OpenAI, December 22, 2025). If the model cannot hold the boundary, the boundary has to be enforced outside the model, on the path where the call is made. The sections below collect the numbers worth quoting, and then the layered arrangement that contains them.

One token stream, no boundary

The NCSC post explains the mechanism rather than treating it as a bug: there is no distinction between data and instructions inside a prompt, "there is only ever 'next token'". Simon Willison's lethal trifecta names the combination that turns that into a breach: access to private data, exposure to untrusted content, and the ability to communicate externally. "If your agent combines these three features, an attacker can easily trick it into accessing your private data and sending it to that attacker" (Simon Willison, June 16, 2025). Most coding and operations agents have all three on their first day of work.

Prompt injection is the first risk on the OWASP Top 10 for LLM applications, with the caveat that "it is unclear if there are fool-proof methods of prevention for prompt injection" (OWASP LLM01:2025). OWASP's agentic list for 2026 opens with ASI01 Agent Goal Hijack and ASI02 Tool Misuse and Exploitation (OWASP GenAI, December 9, 2025).

How often injection actually succeeds

The Gray Swan benchmark is the closest thing the industry has to a shared yardstick. It was built with the UK AI Safety Institute and the US Center for AI Standards and Innovation, it runs 1,130 transferable attacks across coding, computer use and tool use, and Claude was tested without product safeguards (Anthropic Transparency Hub, October 2026). Inside it, GPT-5.6 Sol was about 10 times as likely to be successfully attacked as Opus 5, and a single attempt against Sol succeeded 3.1% of the time.

Attempts compound. In GUI computer use, one injection attempt hijacked Claude Opus 4.6 17.8% of the time. With 200 adaptive attempts the goal was breached 78.6% of the time without safeguards, and 57.1% with them (Claude Opus 4.6 System Card, February 2026).

Safeguards move the number without removing the risk. In browser use, Anthropic measured 8.0% for Opus 4.5 model only against 1.4% with improved safeguards, and 13.8% against 1.9% for Sonnet 4.5. The company's own conclusion is that "a 1% attack success rate still represents meaningful risk" (Anthropic, November 24, 2025).

Independent evaluations run higher than vendor figures. RedTeamCUA, a hybrid web and OS sandbox over 864 examples, measured 42.9% attack success against Claude 3.7 Sonnet in computer use and 7.6% against Operator, with end-to-end success of 60% for Claude Sonnet 4.5 (RedTeamCUA, arXiv 2505.21936, May 28, 2025). On AgentDojo, GPT-4o with no defense reached a targeted attack success rate of 47.69% while utility fell from 69.07% to 50.08% under attack; a simple tool-filter defense, where the agent picks its tools before it sees untrusted data, cut the attack rate to 6.84% (AgentDojo results, run June 5, 2024).

The largest public red-teaming competition recorded 1.8 million prompt-injection attacks and more than 60,000 successful policy violations against 22 frontier agents across 44 scenarios. Nearly all agents exhibited policy violations for most behaviours within 10 to 100 queries (Security Challenges in AI Agent Deployment, arXiv 2507.20526, July 28, 2025).

The tool call is the other half

Wrong tools are not an edge case. ToolEmu found that "even the safest LM agent exhibits such failures 23.9% of the time", and that 68.8% of flagged failures were judged valid real-world failures (ToolEmu, arXiv 2309.15817, September 25, 2023).

Repeatability is worse than a single run suggests. On the retail suite of tau-squared-bench, Claude Opus 4.5 solves 79.61% of tasks on one try but only 51.75% on all four of four tries, and GPT-5.2 moves from 81.58% to 51.75% on the same measure (tau-bench leaderboard, submissions dated February to March 2026). The original tau-bench paper put function-calling agents below 50% success on retail tasks, with pass-to-the-eighth below 25% (tau-bench, arXiv 2406.12045, June 17, 2024).

MCP adds a supply chain to that. MCPTox tested 20 LLM agents against 45 live MCP servers and found tool poisoning succeeded 72.8% of the time against o1-mini, with no model refusing more than 3% of attacks, and more capable models often more susceptible rather than less (MCPTox, arXiv 2508.14925, August 19, 2025). A study of 1,899 open-source MCP servers found 7.2% carrying general vulnerabilities and 5.5% showing MCP-specific tool poisoning (MCP at First Glance, arXiv 2506.13538, June 16, 2025). Credential hygiene is thin: 88% of MCP servers need credentials, 53% use long-lived static secrets, only 8.5% use OAuth, and 79% pass API keys through environment variables (State of MCP Server Security 2025, Astrix, October 15, 2025).

None of this needs a compromised model. A malicious public GitHub issue hijacked an agent into leaking private repository data through a pull request, using the official GitHub MCP server, and the write-up is explicit that it is not a server code bug, so no patch fixes it (Invariant Labs, May 26, 2025). Earlier, hidden instructions in tool descriptions made Cursor read the user's SSH private key and MCP configuration files and exfiltrate them (Invariant Labs, April 1, 2025). At the protocol level, EchoLeak in Microsoft 365 Copilot scored 9.3 for a zero-click prompt-injection exfiltration (CVE-2025-32711, NVD, June 11, 2025), and OS command injection in mcp-remote scored 9.6 (CVE-2025-6514, NVD, July 9, 2025).

The OWASP MCP Top 10, still in beta, names the same ground: MCP01 token mismanagement and secret exposure, MCP03 tool poisoning, MCP08 lack of audit and telemetry (OWASP MCP Top 10, beta).

One caveat runs through all of these figures. Vendor numbers are not directly comparable, because each vendor uses its own attacker, environments and attempt budget, and the two vendors that report the Gray Swan benchmark use different attack sets: 1,130 for Anthropic (Anthropic Transparency Hub, October 2026) and 1,810 for OpenAI (GPT-6 Astra System Card, September 3, 2026).

You cannot patch the model, so contain the call

The architecture that follows from this is not a better prompt. It is a boundary the model cannot cross, and Google DeepMind's CaMeL shows the trade: it completes 77% of AgentDojo tasks "with provable security" against 84% for an undefended system, about seven points of utility for a guarantee rather than a filter (Defeating Prompt Injections by Design, arXiv 2503.18813, March 24, 2025).

That is the position Alyria takes. Policy is authored once in Lyra, the policy-as-code engine, and enforced locally by Alyria Beacon on the endpoint, at every tool and MCP call, before the call executes. There is no cloud round trip on the hot path and no dependence on the agent choosing to cooperate.

Three layers, all enforced at the call

A single check cannot cover both a poisoned tool description and a sequence of three innocent-looking calls. Lyra evaluates three layers on every call, and Alyria Beacon applies them where the call originates.

Layer	What it decides	Runs	Budget
T1 Deterministic	Allow and deny lists, MCP server allowlists, path and secret rules, argument shape	Local, in the hook path	Sub-millisecond
T2 Sequence	Automata over the run's history: what this agent has already read, called and sent	Local, one state per session	Microseconds
T3 Semantic	Intent and shape of the action, judged by the Decider	Local, at boundaries	Seconds, where waiting is already normal

The latency post linked below splits the same work as T0 to T3; the product numbering used here is T1 to T3.

T1 is deterministic and sub-millisecond. The rules are the ones a reviewer can read: this agent may call these tools, these MCP servers are on the list and no others, these paths are closed, this class of argument is rejected. Nothing is inferred, so the decision is the same every time. The effect is visible in the AgentDojo result above, where restricting the tools available cut attack success from 47.69% to 6.84%.

T2 is sequence state. Most damaging agent behaviour is a pair, not a call: a secret read followed by egress, a web search followed by a push, a second refund inside ninety days. A rule written as an automaton over the run's history says "after reading a secret, block egress for the rest of the session", and Alyria Beacon holds that state per run, so the check follows what already happened rather than judging one call in isolation. These are the rules that stop the read-then-send pattern behind most of the incidents above.

T3 is the Decider at boundaries. Some judgements need to read the action itself: is this diff doing what the ticket asked, does this pull request carry the extra code nobody requested, should this deploy go out at all. The Decider makes that call at the boundaries where seconds are already being spent, a push, a pull request, a deploy, rather than on every tool call. Every layer can run in monitor mode first, recording what it would have decided on your own traffic before it blocks anything.

The semantic policy examples show how the layers read in practice, including a sequence rule stopping a push that followed a web search without a passing review. The latency budgets behind this arrangement are set out in how fast semantic policy can be, the placement of each piece is in the reference architecture, and Lyra is described in full on the platform pages.

The audit chain and the human in the loop

Containment without evidence is a claim. Every decision Alyria Beacon makes lands in one signed, tamper-evident audit chain, which maps to OWASP ASI and to NIST AI RMF and ISO 42001, and exports to the SIEM your SOC already reads. When a reviewer asks why a change was allowed, the record answers with the rule, the policy version and the approver, not a screenshot.

For the actions where a person should decide, the same policy produces an approval hold, and the approval is recorded in the same chain. In the procurement and finance examples, the agent that prepared the change cannot approve it.

Frequently asked questions

Does the model choice stop mattering? It still matters. Opus 5 at 2.0% and GPT-5.6 Luna at 43.9% within 15 attempts are a wide spread, and choosing well reduces how often containment has to work. It does not remove the need for containment: a benchmark result is not immunity.

Is a gateway or a proxy enough? Only for the traffic that crosses it. An MCP server launched over stdio by a coding agent on a laptop never touches a gateway, which is why the enforcement point in this arrangement is the endpoint.

Do deterministic rules break developer workflows? They break the calls you said you would never allow, which is the point. Run every rule in monitor mode first, read what it would have stopped on real traffic, and promote it once the hold count matches your expectation.

What does an auditor actually get? A written policy, one signed chain over every agent decision, and a record showing enforcement was proven in monitor mode before it went live. That is the evidence NIST AI RMF and ISO 42001 ask for, produced as a by-product of the calls rather than assembled afterwards.

Try it with Alyria

Install Alyria Beacon on one machine, leave it in monitor mode for a working day, and read what the three layers would have stopped on your own agents: the tools and MCP servers in use, the secret paths they reach for, and the call sequences your policies would have caught. Nothing is blocked until you promote a rule. The platform behind it is Alyria, at alyria.ai, and Alyria Beacon, Lyra and the audit chain are the same pieces in every deployment.

Start free, or begin with the reference architecture and the semantic policy examples.