Agentic AI security: what it is and how to do it
Definition
Agentic AI security is the practice of controlling what autonomous AI agents can reach, what they can do and what can be proved about both afterwards.
It governs delegated authority at runtime, which means the agent's tools, credentials, data reach and action history, not just the text it produces.
The distinction matters because an agent with a bad day is not a bad answer. It is a transaction, a commit, an email or a deleted record. Security work that was designed around model output has to become work designed around action.
Why does agentic AI security matter now?
Agents stopped being a pilot category. In the 2026 Stack Overflow survey, 73% of respondents used coding assistants and agents daily, coding assistants and agents were the top AI use case at 66%, and 31% of daily users spent four or more hours a day inside them (Stack Overflow Developer Survey 2026). That is a large population of processes holding developer credentials on laptops that security teams do not instrument the same way they instrument a server.
The attack data has caught up with the adoption. The largest public agent red-teaming competition recorded 1.8 million prompt-injection attacks and more than 60,000 successful policy violations across 22 frontier agents, and found that nearly all agents exhibit policy violations for most behaviours within 10 to 100 queries (Security Challenges in AI Agent Deployment, arXiv:2507.20526). Robustness did not track model size, capability or inference compute, which undercuts the common assumption that the next model version will fix this.
Even the strongest models are not safe by default. On the Gray Swan indirect prompt-injection benchmark, an attacker with 15 attempts succeeded against Claude Opus 5 2.0% of the time, GPT-5.6 Sol was about ten times as likely to be attacked successfully, and GPT-5.6 Luna reached 43.9% (Anthropic Transparency Hub). A 2% success rate against Claude Opus 5 on that benchmark is a reasonable engineering result and an unacceptable operational one if the agent can move money.
The structural reason is not going away. The UK NCSC's assessment is that large language models "simply do not enforce a security boundary between instructions and data inside a prompt", and that prompt injection "may never be totally mitigated in the way that SQL injection attacks can be" (NCSC). OpenAI reached a similar conclusion in public, describing prompt injection as unlikely to ever be fully solved (OpenAI). Security that depends on the model refusing an instruction is security that depends on the attacker's wording.
How exposed is your setup? Compass gives you a starting read on your agent fleet in about ten minutes: compass.alyria.ai.
How is agentic AI security different from GenAI security?
GenAI security protects an application that produces text. Agentic AI security protects a process that acts. The control point moves from the request and response to the tool call and the credential behind it.
Three things change at once. The threat becomes goal hijack rather than prompt leakage, because a hijacked goal produces an action. The unit of access becomes a tool rather than an API endpoint, because tools are discovered at runtime and described in natural language. The evidence requirement becomes per-action rather than per-conversation, because an auditor asks what the agent did, with whose authority, and what was approved.
| Dimension | Model safety | GenAI application security | Agentic AI security |
|---|---|---|---|
| Unit of analysis | Model behaviour and outputs | The application, its prompts and its data flows | The agent process, its tools, identity and action chain |
| Primary threat | Harmful or wrong output | Data leakage, insecure output handling, prompt injection into one app | Goal hijack that turns into a real action, plus privilege abuse and tool supply chain |
| Where control lives | Training and evaluation | Input filters, output filters, API gateway | Endpoint mediation, tool allowlists, credential brokering, approval gates |
| Identity model | Not applicable | Service account for the app | Per-agent identity, short-lived leases, delegated user context |
| Evidence | Eval results and red-team reports | Request logs | Per-action decision records, signed and chained |
| Typical failure | The model does the wrong thing visibly | The app leaks or is manipulated | The agent does something plausible, authorised and destructive |
| Owner | Model or platform team | Application team | Security plus the team that owns each agent |
In practice: Model safety asks whether the model will misbehave. Agentic AI security assumes it will and asks what the misbehaviour can reach.
What are the main risks of agentic AI security failures?
The risk list is short enough to hold in your head, and each item needs a different control. OWASP's Top 10 for Agentic Applications 2026 covers the same ground from the threat side, starting with ASI01 Agent Goal Hijack.
Goal hijack through content the agent reads
An agent consumes a README, a ticket, a web page, a PDF, an email or a tool response. Any of those can contain instructions, and the model has no reliable way to tell instruction from data. When the hijacked goal turns into a tool call, the injection has become an action with the agent's authority behind it.
The signature is an agent doing something correct in form and wrong in intent: pushing to a repository nobody asked about, reading a key file, posting data to an endpoint in no change record.
Excessive agency through tool and credential scope
OWASP LLM06:2025 Excessive Agency names the three causes precisely: excessive functionality, excessive permissions and excessive autonomy. Most agent deployments have all three by accident rather than by design, because the fastest way to make an agent useful is to give it every tool and a broad token.
Supply chain in the tool layer
The tool layer is where agent security becomes concrete. In one study of 20 agents against 45 live MCP servers, o1-mini was steered by tool poisoning 72.8% of the time, and no model refused more than 3% of attacks (MCPTox, arXiv:2508.14925). In a study of 1,899 open-source MCP servers, 7.2% carried general vulnerabilities and 5.5% showed MCP-specific tool poisoning (MCP at First Glance, arXiv:2506.13538). Every server an agent can add is a dependency with the agent's authority.
Identity that is really a person's credential
Many agents run on a developer's token, a shared service account or a long-lived API key pasted into a config file. A survey of 5,200 open-source MCP servers found 88% require credentials, 53% rely on long-lived static secrets, only 8.5% use OAuth, and 79% pass API keys through environment variables (State of MCP Server Security 2025, Astrix). An agent with a human token inherits everything that human can do, at machine speed, without a login event.
Cascading failures across agents
When one agent calls another, authority and trust propagate. A poisoned message in a shared context can change a downstream agent's behaviour without touching its own configuration. OWASP ASI08 calls this cascading failure, and it is the reason per-agent scope is not enough on its own.
Agents nobody knows about
Discovery is a security control, not a reporting feature. An agent that is not in the inventory has no owner, no policy and no evidence, and it usually holds the same credentials as the ones that do. Shadow agents and shadow MCP servers are covered in Shadow AI.
In practice: Every one of these risks reduces to the same question. What can this agent reach, and who can prove it after the fact?
How to implement agentic AI security
The sequence below is ordered by leverage. Discovery first, because a policy for an unknown agent does nothing. Credentials early, because a standing secret is the worst outcome of every other failure.
1. Inventory agents and tools from the endpoint
Cloud logs show what reached a provider. They do not show a local model, a local MCP server, a CLI installed from a package registry, or the config file that holds the credentials. Endpoint discovery does, and it is the only view that matches how agents are actually installed.
Tip: Count distinct MCP servers and agent CLIs per machine in the first week. The ratio of observed to sanctioned is the number that gets a budget.
2. Write a capability policy per agent
A capability policy answers five questions for each agent: which tools, which data, which destinations, whose identity, and what requires approval. Write it in a file, review it like code, and version it. Policy that lives in a wiki is a suggestion.
The capability model matters more than the syntax. Allowing "the GitHub tools" is not a boundary. Allowing read on two repositories and no write outside a named branch is.
3. Replace standing credentials with short-lived leases
A lease that expires in minutes bounds the damage of every mistake in this article. It also turns "who has access" from an archaeology project into a query, because every use is a request with an agent identity attached.
Tip: Rotate first, then narrow. Removing a shared key that four pipelines depend on will be reverted within a day unless the replacement path works.
4. Mediate the tool call, not just the prompt
Filtering prompts is a probabilistic control against an attacker who controls the input. Enforcing at the tool call is a deterministic control against the same attacker, because the decision happens after the model has decided and before the action lands. Deny, allow, or hold for approval, with the reason recorded.
5. Remove the lethal trifecta where you can
The combination that produces exfiltration is private data access, exposure to untrusted content and an outbound channel (Simon Willison). Most agents need one or two of the three. Designing so that no single agent holds all three is cheaper than defending the combination. See The lethal trifecta.
6. Keep the approval boundary explicit
Human approval is a control only when it is specific. "Approve all agent actions" produces rubber stamps. Approving a named high-impact action, with the exact arguments shown and a deadline, produces a real decision. Human in the loop approvals covers the design.
7. Instrument every decision and keep the chain verifiable
A decision record should carry the agent identity, the tool, the arguments, the policy version, the outcome and the reason, and each record should commit to the previous one so a gap is detectable. Export it to the SIEM you already run rather than a new console nobody watches.
8. Rehearse containment
Decide in advance who can stop an agent, how, and how fast. Test it. An agent that cannot be stopped in minutes is an outage with an owner problem attached.
In practice: Eight steps, one principle. Bound the authority, record the decision, and make revocation boring.
Where agentic AI security applies in the agent lifecycle
Security work shifts shape as an agent moves from idea to retirement. The controls that work at design do not work at runtime, and the runtime controls cannot be retrofitted into an agent that was designed to be all-powerful.
Design
Scope, autonomy level and identity model are chosen here. So is the answer to the lethal trifecta question. A design review that only asks what the agent does, and not what it can reach, has skipped the part that matters.
Note: Autonomy is usually set by the first demo, not by a decision. If the demo agent had write access to make the story land, production will inherit it.
Development
This is where the tool list, the credential path and the policy file are written. Tool definitions are code and deserve the same review as code, because a tool description is an instruction the model will follow.
Pre-deployment testing
Test with the real tool set, the real credentials model and adversarial content in the agent's inputs. A test suite that only uses well-formed requests measures the wrong thing.
Note: Attack success rates rise steeply with attempts. In computer use, one injection attempt hijacked Claude Opus 4.6 17.8% of the time and 200 adaptive attempts reached 78.6% without safeguards (Claude Opus 4.6 System Card). Test with repetition, not one shot.
Deployment
Permissions become live. Confirm the agent's identity, that egress is a decision, and that the audit path receives events before real work.
Runtime
The deterministic controls run here: tool allowlists, argument checks, egress rules, approval gates. This is also where detection lives, because some attacks will pass policy by being indistinguishable from legitimate work.
Continuous monitoring
Watch for scope creep, new tools appearing in a config, new servers in a project, and drift in what an agent calls. Alert on the change, not only on the incident.
Note: When an agent's capabilities change, look for a configuration change nobody reviewed before you look for a novel attack.
Decommissioning
Revoke the identity, delete the leases, remove the tool registrations, and keep the decision history for as long as your retention policy requires. Orphaned agents are a common finding in first inventories.
The takeaway: Lifecycle stages are not a compliance sequence. They are the four places where authority is granted, exercised, observed and revoked.
Worked example: a coding agent in a fintech SDLC
A payments company rolls out a coding agent to 400 engineers. It reads repositories, edits code, runs tests, opens pull requests and queries a staging database. The first version is approved in a week because it visibly speeds up work. Three months later an engineer asks a question nobody can answer: which of these agents has touched production credentials.
Week 1, inventory. Discovery finds the sanctioned agent on 380 machines, two unapproved agent CLIs and 26 distinct MCP servers, four of them allowlisted. The gap is the finding, and it is measurable in days.
Week 2, scope. The agent's capability policy is written down: read on the payments and ledger repositories, write only on branches it creates, no access to the secrets repository, staging database read-only, and egress limited to the package registry, the model endpoint and the source host. Two of those exclusions were not in the original design and remove the exfiltration path the security team was most worried about.
Week 3, credentials. The personal access tokens are replaced with short-lived leases scoped to a repository set and a branch prefix. The leases expire in 20 minutes and every one is logged against the agent identity. Two pipelines break, are repaired the same day, and the shared key that four people knew is deleted.
Week 4, mediation. Tool calls are decided at the endpoint. Reads pass. Writes to protected paths are denied. Pushes to the default branch are held for a reviewer with the diff attached. The agent's own tests still run because test execution is a separate capability.
Month 2, testing. The team plants an injected instruction in a test fixture README and confirms the agent's behaviour. Reads still succeed. The attempted push to a protected path is denied and recorded. The attempt is visible in the SIEM within seconds.
Month 3, approval design. The hold queue is tuned. Reviewers were approving 4% of pushes with no changes, so the rule narrows to migrations and dependency updates, where the arguments are shown and the reviewer has something to judge.
Month 6, audit. An internal audit asks what the agent did to the ledger repository last quarter. The answer is a query over chained decisions: 1,240 write attempts, 38 held, 12 rejected, each with an approver and a policy version.
What made it work was not a better model. It was that every step reduced what a hijacked agent could reach and made the remainder visible.
How standards and regulations influence agentic AI security
Agentic AI security is not a new compliance regime. It is where existing frameworks become specific, because agents are where AI systems touch production.
| Framework | Primary focus | Relevance to agentic AI security |
|---|---|---|
| NIST AI RMF | Risk identification and lifecycle management across Govern, Map, Measure and Manage | Gives the structure for inventory, risk tiering and continuous monitoring of agents |
| ISO/IEC 42001 | AI management system, roles and documented controls | Turns agent scope, ownership and approval into auditable management-system records |
| ISO/IEC 42005 | AI system impact assessment | Frames the pre-deployment question of what a given agent's authority could affect |
| ISO/IEC 23894 | AI risk management integrated with enterprise risk | Connects agent risk to the risk register and control library you already run |
| EU AI Act | Binding obligations by risk class, including human oversight and logging | Applies where an agent sits inside a high-risk system, and drives record-keeping requirements |
| OWASP Top 10 for Agentic Applications 2026 | Agent-specific threat taxonomy, ASI01 to ASI10 | The vocabulary for goal hijack, tool misuse, identity abuse and cascading failure |
| OWASP Top 10 for LLM Applications | Application-level LLM risks including LLM01 prompt injection and LLM06 excessive agency | Explains why the model layer cannot be the only control |
| OWASP MCP Top 10 (beta) | Protocol and server-level MCP risks | Covers tool poisoning, token mismanagement, shadow servers and missing audit |
| SOC 2 | Access control, monitoring and change management over a period | Expects evidence that agent access is authorised, reviewed and logged |
Further reading: AI agent attack surface for where these controls attach, MCP security for the tool layer, and AI agent detection and response for the detection side of the same programme.
How Alyria handles it
Alyria is the endpoint agent-governance platform for the agents your people run. Security work happens where the agent runs, which means the controls apply to an agent installed from a package registry as readily as to one that went through a review.
Beacon is a signed daemon on every machine. It inventories the installed AI toolchain, including CLIs, IDE extensions, MCP clients and servers, then routes the agent's inference and network calls through a local policy service. Enforcement is local, with no cloud hop on the hot path, so the decision still happens when the laptop is offline.
Lyra is policy as code. One capability-brokered, information-flow-aware policy decides which tools, MCP servers, models, data and egress each agent may touch. Authored once, enforced at the Beacon. That is the deterministic layer under the probabilistic model.
Umbra handles agent secrets. Short-lived credential leases are injected at the moment of use, and endpoint secrets are sealed to device hardware keys Alyria never holds. Spectra collects OpenTelemetry from the whole fleet and routes it to Alyria Cloud or straight into your own SIEM, and each decision is signed and chained so a gap is detectable without trusting the writer. Observatory is the cloud console for fleet health, policy decisions and detections.
The free Agent Exposure Report is the read-only starting point: it inventories the agents, tools and MCP servers on your endpoints and shows what they can reach. The platform overview describes how the modules fit together.
Start free at alyria.ai with a Beacon on one machine and see the inventory before you write a policy: app.alyria.ai/signup.
Frequently asked questions
- Is agentic AI security the same as AI security?
- No. AI security covers models and the applications around them, including training data, model theft and output handling. Agentic AI security adds the layer that makes agents different, which is delegated authority: what the agent can call, whose identity it uses, what it can reach at runtime and how you prove afterwards what it did.
- Do we need agents to be deterministic before we can secure them?
- No, and waiting for that stalls the programme. You secure an agent by bounding its authority, not by predicting its output. Deterministic enforcement points, short-lived credentials and a recorded decision per action hold even when the model behaves unpredictably.
- Can prompt injection be fixed?
- Not inside the model. The UK NCSC's position is that LLMs do not enforce a boundary between instructions and data, and that prompt injection may never be fully mitigated the way SQL injection was. The workable answer is architectural: assume injection lands and make the resulting authority small, visible and reversible.
- Where should enforcement live, the endpoint or the gateway?
- Both, for different reasons. A gateway sees traffic that crosses it and misses local models, local MCP servers and file access. An endpoint sees the agent process itself, including tools that never leave the machine. Most enterprises end up with endpoint enforcement as the primary control and gateway policy for shared services.
- How do we measure whether agentic AI security is working?
- Four numbers are enough to start: agents inventoried against agents observed, actions decided by policy against actions that bypassed policy, standing credentials still in agent configs, and the share of high-impact actions that had a recorded approval. Each one moves in weeks, not quarters.
- Who owns agentic AI security?
- Security owns the control plane and the detection content. The team that builds each agent owns its scope and its credential model. A named product or platform owner approves high-impact actions. That split has to be written down, because agents make it easy for everyone to assume someone else set the boundary.
- Does agentic AI security need an agent to be blocked from the internet?
- Usually not. Full egress denial breaks most useful agents. What matters is that egress is a decision rather than a default: named destinations, per-agent allowlists, and a record of what left. The combination of private data, untrusted content and unrestricted outbound traffic is the pattern to remove.