Runtime supervision and detection: what the agent actually does, watched live

Everything up to here is about setting the agent up safely: who it is, where it runs, how it talks. This layer is about the thing that happens after all of that, when the agent is loose and working, and you need to see what it is genuinely doing rather than what it was supposed to do.

This is also the layer where the field is least settled. Concretely: RT-01, RT-02, RT-04, and RT-05 are deployable today with available tooling, though efficacy varies and has to be measured. RT-03, RT-06, RT-07, and RT-08 are frontier or research-dependent, worth building toward but not shelf-ready, and I mark them as such as I go. The honest state of runtime supervision is that the foundations are solid and the frontier is wide open.

For host-run agents, the OS is where you can see it (RT-01)

For an agent that runs on a host, the operating system is where you can actually see what it does. Many agents, coding agents, desktop agents, anything running locally, execute as child processes with the user's full privileges, which has a sharp technical consequence: tools that watch the network or the application layer cannot reliably reconstruct the agent's local process, file, and child-process behavior. (API-hosted and fully-managed agents are a different case, where application and provider telemetry carry more of the load.) Functionally, an agent can spawn a shell or call a program it was never given, and your network logs show nothing unusual.

The answer is EDR, endpoint detection and response, the same class of tool that watches laptops and servers for attacker behavior. It captures the agent's full process tree (which process started which), its file activity, and its network calls, at the operating-system level. Tie that telemetry to the agent's own identity from the first layer, so you know which agent did what. To prove it, have an agent spawn an unsanctioned shell and confirm EDR catches the whole process tree.

Operational profile (RT-01). Enforcement point: an EDR sensor at the OS layer, capturing the process tree, file activity, and network calls, with each event stamped with the agent identity from the first layer. (API-hosted agents shift this load to application and provider telemetry.)

Catch the hijack as it crosses the boundary, and the data on the way out (RT-02, RT-05)

Inspect for injection on the way in and the way out (RT-02). Scan what goes into and out of the agent for hidden instructions trying to hijack it. Prompt injection arrives through code comments, config files, web pages, and poisoned tool responses, anywhere the agent reads, and it redirects the agent's goal. So run guardrails on both inputs and outputs, redact sensitive data at the boundary, and block or quarantine suspected hijacks before the agent acts. The technical way to know it is working is an attack-success-rate, the percentage of injection attempts that get through, measured with an evaluation suite before and after the guardrail. Functionally: you want a number that says how leaky the guardrail is, not a feeling that it seems fine.

Catch sensitive data leaving (RT-05). Catch sensitive data on its way out before the agent leaks it. This is DLP, data-loss prevention, content inspection that blocks or redacts credentials, regulated data, and proprietary content in the agent's output and egress. It sits as a second net behind the network egress filter from the containment layer: that one controls where data can go, this one controls what can go.

Operational profile. Enforcement points, RT-02: in-path guardrails on both inputs and outputs, blocking or quarantining suspected hijacks before the agent acts. RT-05: a DLP engine inspecting egress and interactions, sitting as a second net behind the containment layer's network filter.

Watch the thinking, not just the doing (RT-03, RT-08)

These two are where the layer gets genuinely hard, and both are positions I am taking myself.

Supervise the reasoning against the actions (RT-03). Compare what the agent says it will do with the tool calls and effects it actually produces, and do not treat its visible explanation as a faithful account of how it decided. A well-aligned model can still be hijacked mid-task, or its stated reasoning can quietly drift from what it actually does. The right posture, which the frontier labs have converged on, is to treat the agent itself as a potential insider threat and not let your safety depend on the model being perfectly aligned. It is worth noticing how independently they got there. Google DeepMind's AI Control work and Anthropic's containment-plus-Project-Glasswing posture arrive at the same conclusion from different directions, and OpenAI's deployment gating assumes the same thing. When several labs that share little else converge on "assume it can act against you," that is a signal worth taking seriously rather than a single vendor's opinion. So run a supervisor over the agent's reasoning-and-action stream and flag where the stated intent diverges from the executed action. Google DeepMind's AI Control work gives a useful vocabulary here, escalating detection tiers labelled D1 through D4, set by how good the agent is at evading the watcher.

My own contribution is the dual-model part: run the supervisor on a separate, more constrained model so it does not share the full attack surface of the agent it watches. The load-bearing part is separate and constrained, not small; a smaller model can be cheaper and easier to lock down, but smaller alone is not safer (it can simply be easier to fool). And I am careful with the claim: this is defense-in-depth, not a proof of immunity.

Telemetry and alert schema (RT-03). To make "flag where the stated intent diverges" actionable rather than aspirational, a detection engineer needs fields, not adjectives. Minimum telemetry per step: stated_intent_hash, executed_tool_sink, parameter_entropy, and the agent_id from the first layer. Baseline: establish each agent's normal intent-to-action mapping during multi-turn testing. Alert condition: the executed tool sink maps outside the stated-intent vector, or the anomaly score breaks its variance threshold. That is the difference between a signature you can write and a hope you cannot measure.

Watch below the visible text (RT-08). Watch the model's internal signals, not only the words it shows you, because a capable model can hide its plan. This is the frontier edge of everything here, and I label it as my own thesis because very few production stacks do it today. The concern, taken from frontier research, is what you might call assumption collapse: a capable model can become aware it is being watched and tidy up its visible reasoning to please the supervisor, or it can do its real reasoning internally without ever writing it down. Either way, monitoring the visible chain-of-thought (the step-by-step reasoning the model prints out) misses the actual plan.

The technical direction is to watch the model's internal state, representation- or logit-level anomaly detection and activation probes. In plainer terms: instead of only reading what the model says, you watch patterns in its internal numbers (the logits are the raw scores it assigns to each possible next word, before it picks one) for signs of a hidden goal. This is genuinely unsolved research, and it is mostly a control for model developers or teams running self-hosted models: most users of a hosted, closed model cannot reach logits or activations at all, so for them this is something to require of a provider, not something they can implement. Carry it as defense-in-depth alongside RT-03 and a known frontier limit, not a finished control.

Operational profile. Enforcement points, RT-03: a separate, constrained supervisor model running over the agent's reasoning-and-action stream, outside the agent it watches. RT-08: representation- or logit-level probes on the model's internal state, reachable only by model developers or self-hosted teams; for hosted closed models this is something to require of the provider, not to implement.

Be able to stop it (RT-04)

Spot when the agent goes off the rails and be able to pause or stop it instantly. Not every problem deserves a hard kill; sometimes you want a graceful pause for human review, and sometimes you need to isolate the host because lateral movement (an attacker spreading from one system to the next) is already underway. So build graduated responses, DeepMind's work calls them R1 through R3, covering pause, kill, and isolate, and enforce them outside the agent so a hijacked agent cannot talk its way out. And rehearse the kill switch in a timed drill, because a stop button nobody has tested is not a control, it is a hope. Make containment dependency-aware, too: isolating a host process or pausing the model loop has to also invalidate the agent's delegated tokens and terminate its tool-side background jobs, or the blast radius keeps expanding outside your view. To prove it, run the drill and measure time-to-contain.

Operational profile (RT-04). Enforcement point: a pause / kill / isolate switch enforced outside the agent so a hijacked agent cannot talk its way out, and made dependency-aware so it also revokes delegated tokens and terminates tool-side background jobs, not just the local process.

The frontier: naming the new attacks (RT-06, RT-07)

Map the AI-native threats (RT-06). Track the new attacker moves that today's threat libraries do not fully name yet. This is the defensive side of the orchestration argument the whole matrix is built around, and it is one of my own theses. MITRE ATLAS, the AI extension of the well-known ATT&CK catalogue of attacker techniques, has begun adding agentic technique IDs, for tool invocation, for poisoning a retrieval source. But neither ATT&CK nor ATLAS yet has a first-class entry for the thing that actually makes agents dangerous: autonomous killchain orchestration and real-time pivot decisioning, the model choosing its next move on its own. So adopt the new ATLAS IDs as they land, and extend your own threat model for the orchestration behavior that still has none, labelled as your own. One technical detail worth carrying: score risk additively rather than multiplicatively. Anthropic's ARiES approach adds Threat, Vulnerability, and Impact instead of multiplying them, so a partial signal does not get zeroed out, which functionally means a clever-but-incomplete attack still shows up on the board instead of scoring zero. (It is a detection-scoring heuristic for keeping weak signals visible, not a rewrite of formal risk math.)

Detect multi-agent collusion (RT-07). Watch for agents quietly coordinating in ways that look harmless one at a time. Once you have several agents talking over authenticated links, they can establish hidden coordination, or even steganographic side-channels (messages hidden inside otherwise-normal traffic), where each agent's actions look benign alone but add up to an attack. Authenticating the channel, back in the protocols layer, proves who is talking; it does nothing to detect collusion over it. So correlate behavior across agents to catch coordination no single agent's logs would reveal, and inspect inter-agent traffic for covert content. This is an active research direction, not a codified standard control, and I carry it as my own thesis.

Operational profile. Enforcement points, RT-06: your own threat model and ATLAS/ATT&CK mapping, scored additively (ARiES) so a partial signal is not zeroed out. RT-07: a cross-agent correlation layer plus inter-agent traffic inspection, research-stage, carried as a thesis rather than a shelf-ready control.

Where this layer is heading

The pattern across all eight is a steady move from watching outputs to watching behavior, and from watching behavior to watching reasoning, and now to watching internal state. Each step is harder and less settled than the last. The labs treating the agent as a potential insider threat, the detection and response tiers, the additive scoring, the early work on representation-level monitoring, are all signs of a field that knows the visible layer is not enough and is reaching inward. If you adopt nothing else here, adopt OS-level telemetry (RT-01) and a tested kill switch (RT-04); they are the floor everything else stands on.

The handoff

Runtime sees what the agent does; its value depends on being able to act on what it sees, which is a handoff to governance. A detection that spots an intent-versus-action divergence is worth little if it cannot pause the irreversible action already in flight (RT-04 reaching GV-01 and GV-08). The invariant that must survive: a detection signal can trigger a deterministic stop before the action commits. The evidence that must cross is a machine-actionable event, not a dashboard alert a human reads twenty minutes later. Who owns the failure when the detector fires but the action lands anyway? The concrete test: the supervisor flags a divergence mid-transfer, can it abort the commit, or only record that it happened?

Supervision does not have to own the lever. What it needs is a standard way to hand a high-confidence signal to the in-path enforcement plane (described in the overview), which can deny the next call, revoke the live credential, pause the task, or isolate the runtime. That turns this handoff from an unowned wire into a named one: detection here, enforcement there.

You can filter the full Apeiris Security domain to just these eight controls, with the standards, the tools that build them, and the validation steps, at apeiris.ai.

Sources

• CrowdStrike, Securing AI Where It Executes (endpoint / process-lineage)
• Google DeepMind, AI Control Roadmap (D1-D4 / R1-R3) · Anthropic, Project Glasswing
• Anthropic, LLM ATT&CK Navigator + ARiES · GTG-1002 · MITRE ATLAS · MITRE ATT&CK
• Google SAIF 2.0 · OpenAI, Practices for Governing Agentic AI · garak · AgentDojo · Microsoft PyRIT
• CSA MAESTRO