The five signs your AI agent rollout has outgrown its controls

An AI agent rollout has outgrown its controls when agents are doing work that the inventory, the policy, the audit record and the approvals were never designed for. That usually happens quietly, over one quarter, in the gap between a pilot and a fleet. The five signs below are the ones that show up in review meetings, and each one comes with a check you can run in an afternoon.

A rollout that outgrows its controls is not a failed rollout. It is a normal one that reached the stage where the controls matter, and a program that catches it here spends a week on catch-up rather than a quarter on recovery.

The quick win: run the inventory check first

Take five machines from different teams, list every AI CLI, agent and MCP server on them, and compare that list with what your asset register says.

The check takes about an hour. Alyria Beacon produces the same list read-only in about 30 minutes, including versions, MCP servers each client trusts and secrets found in config. The first time this runs on most fleets it finds servers nobody sanctioned, and that is the finding rather than a failure of the pilot. If the list cannot be produced at all, you have sign one and the other four are already true by default.

Sign 1: nobody can name the MCP servers

Inventory that stops at "we use Claude Code and Cursor" is a licence count. The MCP servers are the part that decides what an agent can reach, and they are configured per client, per machine, often by a developer solving a problem at 5pm.

The supply side does not help. A survey of 5,200 open-source MCP servers found 88 percent require credentials, 53 percent use long-lived static secrets, only 8.5 percent use OAuth, and 79 percent pass API keys through environment variables (State of MCP Server Security 2025, Astrix). Servers that hold static keys are servers whose exposure is a data problem rather than an availability problem.

The check. Count distinct MCP servers across the sample and count how many have a named owner. The ratio is your inventory score. If the second number is zero, nothing downstream can be scored.

What to do. Keep the inventory continuous rather than annual, and use it as the input to policy rather than as a report.

Sign 2: policy lives in prompts and settings files

A line in an AGENTS.md that says do not push to main is advice. A vendor settings file is a client-side control on a machine the developer administers. Both are worth having and neither is a security boundary, because the thing being governed is running with the same privileges as the thing being configured.

The check. Ask which decisions are taken outside the model. If the answer is a hook, a settings file and a prompt, the control question scores 1 at best. If the answer includes a decision made by something the agent cannot edit or talk out of, it scores 2.

What to do. Write the rules that matter as policy, evaluate them where the action originates, and keep them in one place across vendors. Lyra is capability-brokered and information-flow-aware, so a rule can name a tool, a resource and a data class, and can depend on what the session already read.

Sign 3: enforcement arrived without a monitor period

This is the fastest way to lose a rollout. A rule turned on in an afternoon blocks a legitimate workflow, the affected team escalates, and the rule comes off. Six weeks later nobody can say which rules are live or why.

The check. Count the rules currently enforced and count how many have a monitor-mode decision record behind them. Rules enforced on a demo rather than on your traffic are the ones that fail first.

What to do. Monitor first, then promote per category on evidence. Alyria Beacon deploys read-only and starts in monitor mode, where it records what a policy would have decided without blocking. Every tenant also receives a managed baseline of rules reconciled into monitor mode, and promoting one to enforce is an explicit admin decision in the console.

Sign 4: the evidence lives in five consoles

An audit question arrives as one sentence: show what the agents did, and what was decided. If the answer requires exports from the model provider, the agent client, the gateway, the pipeline and the SIEM, the honest answer is that there is no record.

OWASP lists the gap in its own right: the MCP Top 10, still in beta, includes MCP08, lack of audit and telemetry, alongside MCP09 shadow MCP servers (OWASP MCP Top 10). Two of the ten risks are about not knowing what happened.

The check. Ask a person who is not on the program team to export the last 30 days of agent decisions and verify them. Time the exercise. Anything over an hour is the finding, and anything that cannot be verified at all is a control at 0.

What to do. Keep one signed, hash-chained record that includes the decision, the rule, the revision and the action, and export it on a schedule so the export path is exercised before it is needed. Spectra routes the same signals to Observatory or to your own SIEM over OpenTelemetry, which keeps the SOC's existing detections in play.

Sign 5: approvals are on every call or on nothing

Both extremes fail. Approving every tool call produces fatigue and then reflexive approval, which is worse than no approval because it manufactures consent on the record. Approving nothing means a high-impact action has no named owner.

The check. List the high-impact actions in the last 30 days: production deploys, payment releases, deletions, merges to protected branches. Count how many have a named approver attached in a system rather than an email thread.

What to do. Place approvals on the specific actions with an owner and a consequence, and record them in the same audit chain as the policy decisions. Trust is conditional in the population you are protecting: 48 percent of developers trust AI output only when they can easily verify it (Stack Overflow 2026). An approval is the moment a person says they verified it, so the record has to survive.

What it costs to leave these alone

Gartner expects more than 40 percent of agentic AI projects to be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls (Gartner). The third reason is the one this list addresses, and it is the cheapest to fix while the rollout is still small.

The technical case is not subtle either. In the largest public agent red-teaming competition, 1.8 million attacks produced over 60,000 successful policy violations across 22 agents (Gray Swan and UK AISI, arXiv 2507.20526). Adaptive attackers make the point about persistence directly: against Claude Opus 4.6 in GUI computer use, a single injection attempt succeeded 17.8 percent of the time, 200 adaptive attempts reached 78.6 percent without safeguards and 57.1 percent with them (Claude Opus 4.6 system card). Safeguards help; they do not remove the need to decide what happens when one fails.

Score the five, then the program

Each sign maps to a control question, and the free assessments score both questions per function in about ten minutes. Start with the track that matches the team that grew fastest: software development, security operations, procurement, customer service or financial operations.

The agentic AI maturity model explains what each level expects of inventory, policy, monitor mode, audit and approvals, so a score and a plan point at the same things.

Frequently asked questions

How many of these signs mean we should pause the rollout?

One, if it is sign 3 or sign 4. Enforcement without a monitor record and evidence that cannot be exported both mean the program cannot answer a question it will be asked. The others are fixable while work continues.

We already have a SIEM. Does sign 4 still apply?

Only if agent decisions reach it. Most fleets have policy decisions in three vendor consoles and no path to the SIEM, which is the gap Spectra closes by emitting OpenTelemetry from every Beacon.

Is a gateway enough for sign 1?

No. A gateway sees what is routed through it, and stdio MCP servers never cross the network. The inventory has to come from the endpoint.

What is the fastest fix for sign 5?

Move approvals to two actions, the merge to a protected branch and the production deploy, and record them where the policy decisions land. Two approvals done properly beat twenty done on instinct.

Do these signs apply outside engineering?

Yes. The same five show up in security operations, procurement, customer service and financial operations, with different high-impact actions. The control questions are identical across all five tracks.

Get a full Alyria maturity assessment

Alyria assesses your agents and controls across functions using Compass (compass.alyria.ai), and delivers a scored maturity report and a roadmap. Talk to us about a full assessment to replace the five checks above with a scored result, or read where you stand first at alyria.ai/assessments.