▸ BLOG

Why observability is not authorization — and why the difference matters for regulated AI

Two different vendors will meet with your Chief Compliance Officer this quarter. One will show a dashboard that renders every agent action, categorizes them, and produces a monthly report. The other will show a runtime that authorizes each agent action against your policy in single-digit milliseconds, before the action touches a system of record, and writes the audit evidence in the same transaction that executes the action.

Both will say they help you deploy AI agents in a regulated environment. Only one is doing the thing your supervisor will ask for during an examination.

This is a category argument, but it is not a semantic one. It has real consequences for how you procure, deploy, and defend AI agents in regulated workflows. If you are a Chief Compliance Officer, a CISO, or a Head of Platform Engineering at a regulated institution, and you are being pitched “AI governance” this year, the difference between observing and authorizing is the difference between a compliance report and a control.

What observability is#

An AI observability platform records the events that an AI system produces during its operation. Model calls with inputs and outputs. Tool invocations. Retrieval hits. Latency, cost, provider errors, retries. Aggregations of these events into dashboards, alerts, and monthly reports. Best-in-class observability adds richer context — decoding embeddings, replaying prompts, running eval suites on captured traces.

This is real work. Datadog, Arize, Weights & Biases, and Fiddler all do it well, in overlapping and complementary ways. If you are a data science team debugging a model in production, or a platform team measuring latency across providers, observability is what you want.

Observability is after. It sees what happened. It cannot prevent what should not happen.

What authorization is#

An authorization runtime evaluates each proposed action against your policy, before that action reaches a system of record, and returns one of three verdicts: permit, deny, or hold for human approval. The evaluation is a function call in the same process as the action; there is no network hop, no queue, no eventual consistency. The verdict is bytes, not a probability. And the record of the verdict is written into the same transaction as the action itself — the two commit together, or neither does.

This is not novel. Cedar (the AWS-published authorization language), Open Policy Agent, and internal policy engines at hyperscalers all do this in their respective domains — IAM, admission control, feature flags. For agents, though, most of what has shipped in the market as “governance” is either a dashboard that reports on what happened, or a policy interface that produces a document rather than a runtime control.

Authorization is before. It decides what is permitted to happen. A denial is not an incident that was noticed — it is a control that fired.

Why this distinction matters more for agents than for classical software#

Classical software runs deterministic code paths that a security team can enumerate. Agents don’t. The model chooses which tool to call, in which order, with which arguments, based on inputs that include untrusted content (a user email, a retrieved document, a scraped web page). That input can and does invert the intended behavior of the agent — the entire prompt-injection literature is a catalog of ways this happens.

The consequence is that an agent’s action set at runtime is not a closed set at design time. You cannot enumerate “the things the agent might do” and pre-audit them. Every action, at the moment it is proposed, is the new thing.

This is precisely why authorization has to happen at the runtime, not at the design surface. The policy that says “no irreversible action from tainted input” only helps if it evaluates on the actual action being proposed, with the actual provenance of the input at the moment, before the action lands in your payments system. A policy that lives in a governance dashboard, evaluated post-hoc against the recorded log of what happened, catches the incident after the money left.

For regulated institutions specifically — this is what the substrate has to do.

Why regulation says the same thing, in its own vocabulary#

DORA Article 5 requires documented governance and control. Article 6 requires an ICT risk management framework. The word “control” in these articles is a specific term of art: a control is a mechanism that prevents an undesired outcome, or detects it and constrains its blast radius. A dashboard that shows you what happened is not a control by this definition. A runtime authorization decision is.

EU AI Act Article 14 requires human oversight for high-risk AI. “Oversight” in this article is not “someone reviews the logs weekly.” It is a specific requirement that a human can intervene in the operation of the AI system — which structurally requires a runtime that can pause for that human, not a system that only reports after the fact.

Article 15 requires accuracy, robustness, and cybersecurity. “Robustness” in this context includes the ability to reject inputs that would degrade the system’s outputs — which is authorization language, not observability language.

The EU AI Act reads as though it was drafted for authorization runtimes because, from the perspective of a regulator, it was. What is called “AI governance” in the market has been the observability posture; what the regulator is asking for is the authorization posture.

The technical objection: “we already log this”#

The most common technical pushback we hear on this framing is: “Our OpenTelemetry setup captures every agent action; our SIEM alerts on anomalies; we don’t need a new runtime, we need better dashboards.”

Two responses.

First, log capture is not authorization. If your OTel trace shows the agent invoked tool.issue_refund at 09:14:22 with amount USD 12,400 — the refund happened. The trace is a receipt. There is no version of a receipt that is also a control.

Second, and this is the harder one: log capture is not audit evidence in the regulatory sense. Your OTel logs can be modified, deleted, replayed, or diverged from the actual sequence of events, and the log itself does not detect this. Your SIEM sits on top of logs; it inherits the trust properties of the logs. When an examiner asks “did this happen exactly this way, and was it authorized to happen exactly this way,” a log with no cryptographic linkage does not answer.

The evidence a regulated institution needs is a signed, hash-chained, append-only record — the audit chain — where each entry is bound to the one before it, and any modification, deletion, or reordering is detectable by an off-vendor verifier. Observability doesn’t produce this artifact; a runtime that commits execution and evidence together does.

The organizational objection: “our compliance team is happy with dashboards”#

This one is harder to answer with technology, because it is an organizational statement. But we have observed a pattern:

A compliance team that is not yet under regulatory examination is happy with dashboards, because dashboards are what they have and what they’ve used for other systems.

A compliance team that is currently under regulatory examination — with an on-site examiner asking for evidence of specific decisions on specific dates — is not happy with dashboards. They are asking their platform team for a way to produce the artifact the examiner is asking for. The gap between “we have the log” and “we can produce the signed byte-string that the examiner will verify offline” is where the pain is.

This pattern is why the buyers currently moving fastest on the runtime substrate are the ones with active examinations, not the ones with the fanciest AI programs. The purchase driver is regulatory teeth, not model capability.

What Orkena is doing about this#

Orkena is a Governed Agent Runtime — a substrate that combines the durable executor and the authorization engine and the hash-chained ledger into a single system. The four properties that define the category are: authorization before execution, evidence in the same transaction, deterministic replay, and evaluation-gated change control. All four have to be in one system for the assurance to be structural rather than aspirational; separating them into vendor products creates seams, and the seams are where the assurance leaks.

The technical case for building this as a single system is in the technical brief. The category argument is in the category essay. The design partner programme for 2026 is at request a demo.

What to ask when you’re being pitched “AI governance” this year#

Two questions, at the top of the technical conversation:

1. Can you deny an action before it executes, and can you show me the denial in a ledger event that commits with the execution attempt?

If the answer is “we capture the attempt in our audit log and alert on it,” you are being pitched observability. Reasonable technology, wrong category for your regulatory posture.

2. Can I run a verifier on my own machine against your audit bundle, without your system being online, and get a byte-level answer on whether the chain is intact?

If the answer requires calling their API, checking their status page, or opening a support ticket, the audit trail is a report about the audit trail, not the audit trail itself. Also reasonable technology; also the wrong category.

The category exists. The regulation is arriving. The infrastructure decisions your platform team makes this year determine whether you are examined for the substrate in place, or examined for its absence.

---

If you are working on this problem inside a regulated institution, we would like to talk to you. Orkena’s design partner programme has four openings for 2026. Request a demo →

This is the first in a series on Governed Agent Runtimes. The next post takes on the evidence bundle specifically: what a hash-chained ledger commits to, what an Ed25519 anchor signs, and why offline verification is the differentiator that a competitor cannot copy in a quarter.