What Is Policy as Code for AI Agents?

Decisions, not descriptions. A policy that is not evaluated at the moment of an action is a preference, and preferences lose to a determined agent.

Deterministic by construction. The same request and the same task state should produce the same answer, which is what makes policy testable and auditable.

Owned and versioned. Every rule has an author, a reviewer, a version and a record of what it decided.

Attached to the action. A decision stored apart from the action it allowed cannot answer who permitted what.

Why is policy as code so important for AI agents today?

Because the alternative, instructing the model to behave, does not survive contact with untrusted content. The UK NCSC put the structural problem plainly: current large language models simply do not enforce a security boundary between instructions and data inside a prompt, and prompt injection may never be totally mitigated the way SQL injection has been (NCSC, December 2025). Anything that relies on a model obeying a sentence is not a control.

The gap between having a policy and enforcing one is measurable. Across 22 frontier agents in the largest public red-teaming competition, 1.8 million attacks produced more than 60,000 successful policy violations, and the authors reported that nearly all agents exhibit policy violations for most behaviours within 10 to 100 queries (Security Challenges in AI Agent Deployment, arXiv:2507.20526). OWASP's LLM06:2025 names the root causes of excessive agency as excessive functionality, excessive permissions and excessive autonomy, all of which are decided by policy or by nobody (OWASP LLM06:2025).

Bounding the action, rather than filtering the text, is what the evidence supports. On AgentDojo, a simple tool-filtering defence, where the agent chooses its tools before it sees untrusted data, cut GPT-4o's targeted attack success rate from 47.69% to 6.84% while utility stayed at 72.16% (AgentDojo results). CaMeL, an architectural defence that extracts control and data flow from the trusted query, solved 77% of AgentDojo tasks with provable security against 84% for an undefended system (Defeating Prompt Injections by Design, arXiv:2503.18813). Both are policy decisions expressed in code: what may be called, and what may influence what.

How is policy as code different from an AI usage policy document?

An AI usage policy is a statement of intent for people. Policy as code is an executable decision for machines. Both are needed, and neither replaces the other.

A usage document sets expectations: what staff may paste into a chatbot, which data classes are off limits, who approves a new tool. A code policy decides, per call, whether this agent may invoke this tool against this resource with this data and reach this destination.

Dimension	Usage policy document	Policy as code
Audience	People	The enforcement point in front of the agent
Unit of control	A principle or an expectation	A capability with scope: caller, tool, data, destination
When it applies	At training, at review, at audit	On every tool call and model request
Failure mode	Nobody reads it, or nobody can apply it under pressure	A rule that is wrong, which the evidence shows quickly
Change control	Document versioning	Version control, review, tests, staged rollout
Evidence	Training records and attestations	A decision record per action with the policy version
Who owns it	Compliance, usually	Security engineering with the business owner of each agent

The document still matters. It explains why the code says what it says, and it is what an auditor reads first. The mistake is treating it as the control.

How does policy as code express least privilege?

Least privilege is the principle. Policy as code is how it survives contact with an agent that changes tools mid-task.

A role-based grant answers what a service may do over a long period. An agent needs something narrower: what this agent may do for this task, against this resource, for this long. Three translations carry most of that.

Capability instead of role. Grant repo:write on one repository for one branch, not repository write across the organisation. The policy states the capability and its scope in the same rule.

Lease instead of credential. Issue a credential for the task and let it expire. A standing token with a broad scope is the opposite of least privilege even when the role behind it is narrow.

Destination instead of network position. Say which hosts and APIs the agent may reach, not which subnet it sits in. Position is an accident of deployment; destination is the decision.

The failure mode is privilege creep through exceptions. A blocked deployment becomes an exception, the exception becomes a rule, and two quarters later the agent holds what it asked for rather than what it needs. The defence is a review cadence with a named owner for each agent's rule set, and a rule that expires unless someone renews it.

What are the main risks of running agents without enforced policy?

Without enforcement, the risks are not exotic. They are ordinary operational failures with an unusual actor.

Authority that expands quietly

An agent starts with read access to one repository and ends up with write access to three because a ticket was easier than a design review. Without a policy that states the maximum, nothing records the expansion.

Diff the rule set on a schedule. An expansion that shows up in review is a decision; one that does not is a finding.

Prompt instructions mistaken for controls

"Never send email without approval" in a system prompt is a sentence the model is asked to follow, and no published measurement tells you how often it will. The case you were worried about is the one where untrusted content arrives in context and the sentence loses.

Keep the sentence and add the rule. The prompt helps the model behave; the rule decides what happens when it does not.

Undetected sequence attacks

Most real incidents are sequences: a page is fetched, then a push happens; a bank detail changes, then a payment is scheduled. A per-call allowlist sees two permitted actions and no problem.

Record the state the rule needs. A sequence rule is only as good as the events the engine can see.

Enforcement in the wrong place

A gateway that inspects outbound traffic cannot see a local MCP server, a local model, or a tool call that never leaves the process. Policy that only covers egress covers the smaller half of the surface.

Test it from the agent's own process. If the call cannot be seen there, it cannot be governed there.

Decisions that cannot be reconstructed

If the decision is not recorded with the action, an investigator can see that a payment was scheduled and cannot see who allowed it, under which rule, or with what evidence. That is the failure an audit finds first.

Who is responsible for policy as code?

Three roles, and a policy programme that skips one of them stalls.

The security engineering team owns the policy engine, the language, the deployment path and the tests. They are accountable for the mechanism.

The business owner of each agent owns the rule content: what this agent may reach, which thresholds apply and which exceptions are legitimate. They are accountable for the authority.

The risk or compliance function owns the mapping to frameworks and the review cadence, and signs off on the exception register.

An agent without a named business owner cannot have a policy, because nobody can say what it is for. That is the first finding of most reviews.

How to implement policy as code for AI agents

Start with one chokepoint and one rule set, then widen. Trying to write the whole policy first is how these programmes fail.

1. Write the action inventory before the policy. List the ten things agents do that would be an incident if they happened unreviewed: reading a credential file, pushing to main, deleting a production record, calling an unsanctioned server. That list is your backlog, in priority order.

2. Choose one enforcement point and hold it. For agents on a laptop, that is the endpoint, in the same process boundary as the tool call. Treat a gateway as defence in depth rather than the primary control.

3. Express each rule as a capability with scope. Caller identity, tool or model, resource, data class and destination. A rule about a tool name alone is a rule about a name.

4. Decide the approval path inside the rule. Hold for approval with the evidence the approver needs, rather than blocking and creating a side channel. Record the approval as part of the decision.

5. Make the engine deterministic and fast. A policy engine that needs a model call to decide is neither reproducible nor affordable inside an agentic loop where dozens of calls happen per task.

6. Version, review and test the rules. Store policy in the same repository discipline as code, require review, write fixtures for each trigger state, and replay historical calls before deploying a change.

7. Run in monitor mode, tune, then enforce. Review what would have been blocked with the teams doing the work. Add the exception instead of loosening the trigger, then enforce the rules you have proven.

8. Record the decision with the action. Policy id, version, content hash, decision, approver and outcome, in a store the agent cannot edit. Policy without evidence is a claim.

Where does policy as code apply in the agent lifecycle?

Policy written once and left alone drifts out of date within a quarter. It is a lifecycle artefact like any other.

Design

The design defines the maximum authority, and the first rule set is a written version of that decision. If scope is unclear at design, the policy becomes a series of exceptions later.

Development

Policy is implemented here: the capability model, the identity each agent uses, the destinations it needs. Tool choices made during development become the surface the policy has to describe.

Pre-deployment testing

Replay and red-team the rules. Try the sequences you fear, confirm the decision is what you expect, and check that the approval path works when nobody is watching the channel.

Deployment

Rules move to production with a version and a content hash. Confirm the endpoint is receiving the policy and caching it, and confirm the decision record is arriving.

Runtime

Every tool call and model request is evaluated locally before it proceeds. Monitor at first, then enforce. The decision, the reason and the policy version go to the audit chain.

Continuous monitoring

Watch for rules that never fire and rules that fire constantly. The first usually means the policy is not reaching the endpoint. The second means the rule is wrong.

Decommissioning

Retire the rules with the agent, and keep the decision record for the retention period. A rule set nobody owns is a rule set nobody reviews.

Example: a procurement agent under policy as code

A hypothetical worked example. The company, the events and the numbers below are illustrative, not a customer result or a measured outcome.

A retailer deployed a procurement agent that prepares purchase orders from approved requests. Its authority was clear in the design document: orders up to $25,000 with approved suppliers, everything else escalated. The document was not the control.

The team wrote four rules. Orders over $10,000 with any supplier are held for the budget owner. If a supplier's bank details changed in the last 30 days, payment waits for a call-back verification. If this month's orders with one supplier would exceed $50,000 in total, the budget owner approves the aggregate. And a supplier that is not on the approved list is denied outright.

The second rule came from an incident at another company, and it is the one that earned the programme its support. When an invoice mismatch was resolved and the agent scheduled a payment, the rule held it because the bank account had changed six days earlier. A call to the supplier confirmed a fraud attempt that had arrived by email.

The first two weeks were monitor mode, and the review found two noisy rules. The aggregate threshold fired on a legitimate quarterly restock, so the exception was written for recurring suppliers with a signed contract. The bank detail rule produced two false positives on suppliers that had genuinely changed their account, and both were resolved by the call-back process the rule was asking for.

Six weeks later the agent was enforcing eleven rules, the approval queue had a median response of 40 minutes, and every decision carried the rule version that produced it. The agent kept its autonomy and the organisation could say what it had permitted.

How standards and regulations influence policy as code

Policy as code is where most framework language becomes an actual control, which is why the mapping is direct rather than aspirational.

Framework	Primary focus	Relevance to policy as code
NIST AI RMF	Risk identification and lifecycle management	GOVERN is the policy process, MANAGE is what happens when a rule is violated, MEASURE needs the decision record
ISO/IEC 42001	AI management system	Annex A operational control expects documented operating criteria, which a versioned rule set provides
ISO/IEC 42005	AI system impact assessment	The impact assessment defines the thresholds the rules encode
ISO/IEC 23894	AI risk management alignment	Connects agent policy to the enterprise control library rather than a parallel universe
EU AI Act	Binding obligations for high-risk systems	Article 14 human oversight is implemented as an approval rule, Article 12 logging as the decision record
OWASP LLM Top 10	Application security risks for LLM systems	LLM06 Excessive Agency is the permission model the policy expresses
OWASP Agentic Top 10	Agent-specific risks	ASI01 goal hijack and ASI02 tool misuse are the sequence problems policy rules catch
OWASP MCP Top 10 (beta)	MCP-specific risks	MCP02 scope creep and MCP07 authorization are enforced per MCP call
SOC 2	Controls over security and availability	CC6 logical access and CC7 monitoring, evidenced by the rule set and its decisions

How Alyria handles it

Alyria's policy engine is Lyra (alyria.ai): capability-brokered, information-flow-aware policy that evaluates every MCP tool call and model request on the endpoint, deterministically and in sub-millisecond time, before the call proceeds.

Beacon holds the call at a local policy service inside the same process boundary the work runs in, so the decision does not depend on a network round trip. Policy is pulled and cached locally, which keeps enforcement and detection running when the machine is offline.

Rules can reference what the agent is doing and what has already happened in the task, which is the subject of semantic policy. Decisions land in a signed audit chain with the policy version attached, and can be emitted as signed OpenTelemetry into your SIEM. Read Lyra policy in the docs, the semantic policy examples, or how a request is intercepted, decided and recorded.