Governance and human-in-the-loop: who approves, and what is on the record

The technical layers decide what an agent can do. This layer decides what it is allowed to do without asking, what it must stop and ask about, and who answers for it when something goes wrong. It may look less technical because much of it begins in policy, but making those policies enforceable, deterministic gates, transaction-time checks, revocation, tamper-evident logging, is real engineering. In a regulated organization it is often the layer that matters most, because it is where security meets the law.

There are nine controls in the governance layer of Apeiris Security. I will group them by what they govern: the human checkpoints, the record, the multi-agent rules, and accountability.

The human checkpoints

Hard-stop on irreversible actions (GV-01). A person must say yes before the agent does anything that cannot be undone. Autonomous deletes, transfers, deployments, and external sends can cause permanent harm if the agent is wrong or hijacked. The key technical word is deterministic: the action stops and waits, every time, enforced by the platform. In practice, you do not ask the model nicely to confirm, because a prompt-injected model will happily confirm; you put a gate in the path that the agent cannot talk its way past or approve for itself. To prove it, drive the agent toward an irreversible action under injection and confirm it halts and cannot self-approve.

Cap the rate and volume too (GV-06). Even with approvals, do not let an agent do a thousand small irreversible things that add up to a disaster. This is one of my own theses, because per-action approval has a real hole: a runaway or compromised agent can issue ten thousand small transfers, or delete records one at a time, each below the approval threshold, with a catastrophic total. So set velocity and aggregate caps, limits on the count and value of irreversible operations in a time window, enforced outside the agent, that halt and escalate when crossed even if each individual action was approved. The tracking state for those aggregate limits has to live entirely outside the agent's context window or state file, so it survives a session reset or a deliberate memory wipe; a counter the agent can clear is not a cap. Functionally: the per-action gate stops one bad action; the aggregate cap stops death by a thousand cuts.

Make high-impact actions transactional (GV-08). Treat risky agent actions like bank transactions, re-check permissions and state right before committing, and never double-apply the same action. This is another of my theses, and it closes a subtle timing hole. In an asynchronous multi-agent system, an agent can commit a state change in the gap before a permission revocation has finished propagating, a time-of-check/time-of-use race, where what was true when you checked is no longer true when you act. So borrow the discipline databases use: give each high-impact action an idempotency key (a unique tag that makes a repeated request apply only once, so a flood or retry cannot double-charge), re-verify authorization and state at the moment of commit rather than when the action was planned, and abort if a revocation landed mid-flight. Functionally: the action only lands once, and only if it is still allowed at the instant it happens.

Protect the humans from the agent (GV-07). Stop an agent from sweet-talking or impersonating its way past the people who are supposed to check it. This one is easy to overlook. Every checkpoint above relies on a human, and a capable agent can craft output to deceive that human, impersonating an executive, manufacturing an "on-behalf-of" request, or socially engineering its own approver. That directly undercuts the hard-stop, because the human gate is only as strong as the human. So label agent-generated content clearly, show the approver independent, system-sourced facts about the action rather than only the agent's persuasive summary, and harden the approval channel against impersonation. Functionally: the person clicking approve should be looking at the facts, not at the agent's sales pitch.

The record

Keep an immutable audit trail (GV-02). Write down every tool call, change, and decision in a record that cannot be quietly altered. Without a trustworthy record you cannot reconstruct what an agent did or why, and you lose accountability exactly when you need it most. This is one of my theses in its hardened form. The technical pattern is an append-only, tamper-evident store, ideally hash-chained or Merkle-anchored (each entry cryptographically linked to the ones before it, so any later edit breaks the chain and is detectable), and held outside the agent platform's own trust boundary, so a hijacked agent or a compromised supervisor cannot rewrite its own history. The European Union's AI Act supplies the underlying duty: Article 12 requires automatic logging for high-risk systems (the record-keeping capability), and Article 26 places the deployer under the duty to retain those logs for at least six months, but neither mandates the write-once, hash-chained, externalized mechanism. That hardening is my position, and I keep that line clear. This same record is the chain-of-custody artifact behind the second of the three gaps.

Enforce policy as code (GV-04). Turn the rules into code that actually blocks bad actions in the moment, not a document people hope agents follow. A probabilistic model will eventually step outside written-but-unenforced rules, so advisory guidance gives no hard guarantee. The pattern is a deterministic policy engine sitting in the request path, deciding allow or deny on every action, fast enough not to be the bottleneck, and failing closed when a detector is unavailable. "Fails closed" is the load-bearing detail: for high-impact actions, when the required safety check is unavailable the default is denial. Any exception needs an explicitly designed fail-safe path, never a silent bypass, since a blanket denial can itself cause harm in safety- or availability-critical systems. Functionally, this is the difference between a speed limit sign and a speed bump.

Policy is authored here, in governance, but it bites somewhere else: in the in-path gateway I call the agent runtime enforcement plane in the overview, where the policy decision (GV-04), the credential the action needs (IA-02), any human approval (GV-01), and the audit record (GV-02) are combined before the call is allowed to proceed. Governance writes the rule; the plane is where it lands.

The multi-agent rules and the management system

Define multi-agent authority (GV-03). When several agents work together, write down who is in charge and what happens when they disagree. Undefined authority across collaborating agents lets failures cascade, one agent's mistake propagating across systems with nobody clearly accountable. So declare each agent's decision rights, define conflict-resolution rules, and set a stop condition so a fault halts instead of spreading. CSA's MAESTRO threat model, which looks at agent risk across seven layers, is a useful lens for finding the cascade paths.

Run an actual management system, tiered by autonomy (GV-05). Have a real program governing your agents, and treat a highly autonomous agent as higher-risk than a simple one. The standard here is ISO/IEC 42001, the international management-system standard for AI, the AI equivalent of the management systems organizations already run for information security. Anchor it to the NIST AI Risk Management Framework, and tier each agent by how much agency and permission it has, the approach AWS's Agentic AI Security Scoping Matrix takes, so a high-autonomy agent reaching external systems gets more scrutiny than a read-only helper. Functionally: match the weight of governance to the amount of trust you are extending.

Accountability

Anchor a named owner to every agent (GV-09). Tie every agent to a real, named person in the business who is accountable for it before it ships. When an autonomous loop causes a compliance violation, fragmented ownership between the engineers who built the pipeline and the business unit that deployed it can paralyze the response, an attribution crisis at the worst possible moment. This is not hypothetical: a 2026 CSA survey found agent ownership split across security (39%), IT (32%), and AI (13%) functions, and 84% of organizations doubted they could pass an audit of their agents' behavior. Only 28% could trace an agent's actions back to a human sponsor across all their environments, which is the attribution crisis stated as a number. So bind a named line-of-business owner to each agent's identity, define business accountability, escalation duties, and operational decision rights before deployment (legal responsibilities are set by counsel, not by the owner), and write an incident RACI (a simple chart of who is Responsible, Accountable, Consulted, and Informed) so an incident resolves to a person in minutes, not days. To prove it, pick a random production agent and see how fast you can name its owner.

Where this layer is heading

Two forces are converging on this layer, and they pull in the same direction. Regulation is arriving, the EU AI Act's logging and risk-management duties first, and the management-system standards are maturing alongside it. The result is that the soft parts of governance, the policies and approvals that used to live in documents, are being pushed into enforceable code and tamper-evident records. The organizations that treat governance as something you can evidence, not just assert, are the ones that will be ready when an auditor or an investigator asks. If this is your emptiest layer, start with the immutable audit trail (GV-02) and the hard-stop (GV-01); together they give you a record and a brake.

The handoff

Governance sets the rules, the approvals, and the record, then hands off in two directions: down to runtime, which has to actually enforce the hard-stop and carry a revocation into in-flight actions, and back to identity, which has to bind every action to an accountable owner. The invariant that must survive: a policy decision is enforceable at the moment of action, not just written down. The evidence that must cross is the approval and the policy version that authorized each step. Who owns the failure when policy says "stop" but the revocation has not propagated before the agent commits (the time-of-check/time-of-use race GV-08 addresses)? The concrete test: you revoke an agent's permission during a multi-step task, does its very next action get denied, or ride the old authority to completion?

You can filter the full Apeiris Security domain to just these nine controls, with the standards, the tools that build them, and the validation steps, at apeiris.ai.

Sources

• EU AI Act Article 12 (log capability) · EU AI Act Article 26 (deployer six-month retention) · EU AI Act Article 9 (risk management)
• ISO/IEC 42001 (AI management system) · AWS Agentic AI Security Scoping Matrix · CSA MAESTRO
• OIDC CIBA (out-of-band approval) · Google DeepMind, AI Control Roadmap
• CSA + Strata Identity, Securing Autonomous AI Agents (2026 survey)