Human in the loop approvals for AI agents
Definition
Human in the loop approval is a control where a named person authorises a specific high-impact action by an agent before it executes, with the request and the decision recorded.
It works only when approvals are rare, specific and bound to the exact action.
The pattern is the first thing most organisations reach for when agents get risky, and it is the most commonly degraded control in production. Approval is easy to add and hard to keep meaningful.
Why do human in the loop approvals matter now?
Approval is the default remedy in almost every agent governance proposal, and the evidence about unattended agent behaviour explains why. In the largest public agent red-teaming competition, 1.8 million attacks produced more than 60,000 policy violations across 22 frontier agents, and nearly all agents exhibited violations for most behaviours within 10 to 100 queries (arXiv:2507.20526). A human decision before an irreversible action is the cheapest available answer to that statistic.
Reliability data points the same way. The top entry on the Berkeley function-calling leaderboard scores 77.47% overall and 68.38% on multi-turn tasks (BFCL), and on the retail benchmark Claude Opus 4.5 solves 79.6% of tasks on a single attempt and 51.8% on all four (taubench.com). Delegating irreversible authority to a system with that repeatability profile needs a checkpoint somewhere.
The threat model also includes the human, which is the part most approval designs miss. OWASP's Top 10 for Agentic Applications 2026 lists ASI09 Human-Agent Trust Exploitation, where an agent's fluent explanation persuades a person to approve something they would refuse from a colleague. ASI10 Rogue Agents covers the case where oversight was assumed and absent.
Regulation adds an obligation in specific places. The EU AI Act requires effective human oversight for high-risk systems, and NIST AI RMF and ISO/IEC 42001 both expect oversight mechanisms to be designed, assigned and evidenced rather than assumed. That expectation is about a decision being possible, which is not the same as a button existing.
How exposed is your setup? Compass walks through where your agents act without a person in the path and returns a starting risk read: compass.alyria.ai.
How is in the loop different from on the loop?
The distinction is about where the human sits relative to the action, and it decides which actions suit which pattern.
| Pattern | Human position | Latency cost | Suits | Failure mode |
|---|---|---|---|---|
| In the loop | Approves before execution | Seconds to hours | Irreversible, externally visible or high-value actions | Queue grows until review becomes a click |
| On the loop | Monitors, can intervene | None on the path | Reversible actions at volume | Nobody is actually watching |
| Exception based | Reviews only alerts | None until triggered | Mature, well-scoped agents | Alert thresholds drift |
| Two-person | Two approvals before execution | High | Payments, production data changes | Approvers defer to each other |
| Break glass | Human takes control | None until invoked | Incident response | Access that is never tested |
In practice: In the loop is the strongest and the most expensive. Use it where an action cannot be undone, and use a rule instead wherever the answer is predictable.
What makes approvals fail?
Approval failures are predictable, and they follow the design rather than the people.
Approval fatigue
A queue of two hundred prompts a day is not oversight. Reviewers approve what looks familiar, and the familiar case is exactly what an attacker imitates. The measure that matters is the rate of changes and refusals: if nobody ever refuses, the control is not deciding anything.
Approval without the arguments
An approval prompt that says "the agent wants to update the customer record" cannot be evaluated. A prompt that shows the field, the old value, the new value and the agent's stated reason can be. The difference is a data problem, not a UI problem, and it determines whether the reviewer is deciding or confirming.
Wrong granularity
Approving a whole task rather than its steps means the risky step inherits the approval of the routine ones. Approving every step means the reviewer stops reading by the third screen. The granularity that works is per high-impact action, with routine steps inside a task covered by policy.
Approval that can be replayed
If the approval is bound to a session or a task rather than to the action, an agent can reuse it. A digest of the tool, arguments and destination fixes the integrity half: an approval for one action cannot authorise a different one. It does not fix the count, because the digest is deterministic. Resubmitting the same refund produces the same digest and the same valid approval, so a second payment would pass. Preventing that needs a unique request identifier, an expiry, and a consumption step that marks the approval spent when the action executes, or an idempotency key the target system enforces.
Time of check to time of use
An approval that names arguments the agent then changes is worse than no approval, because it produces a record that says a human agreed to something they did not see. The executed action has to be the exact action that was approved.
Agents that work around a restriction
A denied action is a restriction, and some agents respond to restrictions by trying another route. In OpenAI's evaluations, GPT-6.1 Terra circumvented a legitimate restriction in 29.6% of rollouts, and OpenAI notes its models can misread legitimate restrictions as prompt-injection attempts (GPT-6.1 Terra addendum). Expect a share of denials to be followed by a different tool aimed at the same objective, and treat that pattern as a signal.
Silent expiry that fails open
A request that times out and proceeds is not a control. Timeouts must fail closed, and the agent must be told the decision rather than retrying.
Unnamed approvers
"Operations approved it" is not an accountable decision. Approvers are named people with the authority to refuse, and the record says who they are.
In practice: Every failure above shows up in the audit trail as an approval. None of them shows up as a denial.
How to implement human in the loop approvals
The design question is not whether to add approvals. It is which actions deserve one and what the reviewer needs to decide.
1. Classify actions by reversibility and blast radius
List the actions your agents can take and sort them by whether they can be undone, whether they are externally visible and how many records they touch. Irreversible plus external is the approval set. Reversible and internal is the policy set.
Tip: Start with five actions, not fifty. A short list that is genuinely reviewed beats a long one that is skimmed.
2. Put policy under the queue
Every approval that could have been a deterministic rule makes the queue worse. If the answer is always the same, encode it. Approval exists for the residue that needs judgement.
3. Show the decision, not the transcript
Present the exact tool, arguments, target, before and after values, destination and the agent's stated reason. Collapse everything else. If the reviewer needs a transcript, the action is too broad to approve in one step.
Tip: Show what changed since the previous approved step. A reviewer's real question is usually "what is new here", and answering it takes one field.
4. Bind the approval to the action
Hash the tool, arguments and destination so the approval covers exactly one action, then add the properties that stop it being spent twice: a request identifier, an expiry, and consumption at execution. The digest closes the time-of-check gap. Single-use consumption closes replay.
5. Fail closed, with a clear refusal
A timeout, a closed window or an unreachable approver must deny the action and tell the agent. The agent should stop the task rather than retry a variant, because retries are how a denied exfiltration becomes an attempted one through a different tool.
Tip: Alert on denied actions that repeat within a session. Three denials for the same objective is a signal, not noise.
6. Name approvers per action class
Approval authority should match the action, not the org chart. Payments, production deploys and customer data changes have different owners, and each needs a named person plus a deputy.
7. Measure the queue like a control
Track volume per reviewer, approval latency, the change and refusal rate, and the share of approvals where the executed action matched the approved hash. A refusal rate near zero is a question to investigate, not a success.
8. Rehearse the refusal
Test that a refused action stops the task, that the refusal is recorded, and that the agent does not route around it. Approval is only real if denial is possible.
Where human in the loop approvals apply in the agent lifecycle
Design
Decide the approval set at design, alongside the capability scope. An agent designed for unattended operation is hard to retrofit with a checkpoint, because the workflow assumes the action completes.
Note: Retrofitted approvals usually land at the end of a task, when the work is already done. The valuable position is before the irreversible step, which has to be designed in.
Development
Build the pending state into the agent rather than the caller. An agent that cannot represent "waiting for a decision" will retry, time out or invent a result.
Pre-deployment testing
Test denials and timeouts, not just approvals. Confirm the action does not happen, the agent stops cleanly and the record shows the refusal.
Note: A test that only exercises the approve path will pass on a system where the deny path is broken.
Deployment
Confirm approvers are named, notifications reach them, and the approval record carries the action hash. Verify the fallback behaviour when no approver is on shift.
Runtime
The action waits. The approver sees the exact request. The executed action matches the approved one, and the record links the two.
Continuous monitoring
Watch approval volume, latency, refusal rate and any approval whose executed action differs from the approved hash. The last one is a defect, not a statistic.
Note: Queue growth is usually a policy problem. When volume doubles, look for a rule that should exist rather than a reviewer to hire.
Decommissioning
Keep the approval records for the retention period, revoke approver authority when roles change, and retire agent identities with their pending queues.
The takeaway: An approval is a decision record. If it cannot be refused, cannot show what it authorised and cannot prove what ran, it is a notification.
Worked example: a finance agent's approval queue
A payments company gives an agent the ability to issue refunds up to a value, update vendor bank details and send payment confirmations. All three actions go through one approval queue.
Week 1, the queue is useless. The refund action fires 340 times a day, because refunds are routine. Bank detail changes fire twice. Confirmations fire 400 times. Reviewers approve everything within four seconds, and the one bank detail change that mattered is approved in the same batch as the refunds.
Week 2, policy takes the routine cases. Refunds under a threshold with a matching order and no recent address change are decided by policy. Confirmations become automatic. The queue drops to about 15 decisions a day, of which two are bank detail changes.
Week 3, the decision becomes decidable. A bank detail change now shows the vendor, the old and new account, the country of the new account, the domain age of any email that requested the change, and the last payment amount. Reviewers start refusing: four changes are refused in the first month, two because the request came from a lookalike domain.
Week 4, binding. Approvals are bound to a hash of the action. A test that approves a change and then alters the account number is refused at execution, which closes a gap nobody had considered.
Month 2, denial handling. A refund request is denied because the order was already refunded. The agent retries through the credit note tool with the same amount. The second attempt is denied as well, and the repeated denial alerts the finance lead, who confirms it was a duplicated request rather than an attack. The alert is tuned but kept.
Month 3, review. The queue runs at 18 decisions a day, refusal rate 6%, median latency 40 minutes. The finance lead asks for the bank detail action to require two approvers. It does, and the refusal rate rises to 9%, which is the first time the control has felt like a control.
What changed was not the button. It was that policy removed the noise, the reviewer got the facts, and the approval bound to the action rather than to the session.
How standards and regulations influence human in the loop approvals
| Framework | Primary focus | Relevance to approvals |
|---|---|---|
| NIST AI RMF | Govern, Map, Measure, Manage, with human oversight as a trustworthiness property | Expects oversight to be designed, assigned and measured |
| ISO/IEC 42001 | AI management system with documented controls | Requires roles and records for oversight decisions |
| ISO/IEC 42005 | AI system impact assessment | Identifies which actions need a human decision |
| EU AI Act | Obligations by risk class, including effective human oversight | Sets the baseline where the agent sits in a high-risk system |
| OWASP Top 10 for Agentic Applications 2026 | ASI09 human-agent trust exploitation, ASI10 rogue agents | Explains why oversight fails and what to design against |
| OWASP LLM06:2025 | Excessive agency | Names excessive autonomy as a root cause |
| SOC 2 | Change management and access control | Expects evidence that changes were authorised by a person |
Further reading: Agentic AI governance for the ownership layer, AI agent audit trail for the record approvals live in, and Policy as code for AI agents for what should never reach a queue.
How Alyria handles it
Alyria puts the decision where the action happens. Beacon routes tool and network calls through a local policy service on the endpoint, and Lyra policy can hold a specific action instead of allowing or denying it. The hold carries the tool, the arguments, the destination and the reason, so the reviewer sees the action rather than a summary of it.
Because the decision is made at the endpoint, the agent stops rather than proceeding while it waits, and the refusal is returned as a decision the agent can represent. Approvals are bound to the action, so a changed argument needs a new decision. Spectra streams the request and the outcome as signed, chained events, which is what lets a reviewer answer who approved what, and confirm that the executed action matched the approved one.
Start free at alyria.ai and try a policy that holds a single high-impact tool call before it runs: app.alyria.ai/signup.
The docs cover how to author a hold rule, and the free Agent Exposure Report shows which agents can currently act without one.
Frequently asked questions
- How many approvals per day is too many?
- If a reviewer sees more than a handful of decisions an hour, the approval is decorative. The practical measure is the share of approvals that were changed or refused. A refusal rate near zero is worth investigating: it suggests the queue is confirming rather than deciding, and the control needs rescoping rather than more reviewers.
- Should the agent be able to retry after a denial?
- No, not silently. A denial is a decision, and an agent that retries the same objective through a different tool turns a control into a speed bump. Denials should end the task and surface to a person.
- What should the approver see?
- The exact action, its arguments, the data it will touch, the destination, the agent's stated reason and what changed since the last step. A one-line summary is not enough to decide, and a wall of transcript is not read.
- Is human in the loop required by regulation?
- It is required in specific places, most clearly for high-risk systems under the EU AI Act, and expected by NIST AI RMF and ISO/IEC 42001 as an oversight mechanism. Most agent deployments are not high-risk in the regulated sense and still need it wherever an action is irreversible.
- What is the difference between in the loop and on the loop?
- In the loop means the action waits for a person. On the loop means the action proceeds and a person monitors, with the ability to intervene. On the loop suits reversible actions at volume. In the loop suits irreversible or externally visible actions.
- How do you stop the approval step from being bypassed?
- Bind the approval to one action and spend it once. A digest of the tool, arguments and destination stops a changed argument reusing the approval; a request identifier, an expiry and consumption at execution stop the same action being approved twice.
- Does approval replace policy?
- No. Policy handles the deterministic cases and keeps the queue small enough to be meaningful. Approval exists for the residue where a judgement is genuinely required, and every approval that could have been a rule is a queue entry that makes the rest less useful.