AI Agent Governance: Policies, Human Approval and Signed Receipts

Why do AI agents need governance that chatbots do not?

A chatbot answers. An agent acts: it issues a refund, changes a plan, activates a feature, invites a user. The output is no longer text that a person reads before deciding, it is a change in a real system.

That shifts the question from “is the answer good?” to “was this agent allowed to do this, on whose authority, and can we prove it afterwards?”.

What questions must a governance layer answer for every action?

I like the way TrustRail frames it. For every protected operation, the layer has to answer five questions: which agent requested it, who authorised that agent, what limits apply, whether the action was permitted, blocked or held for approval, and what evidence supports the outcome.

Notice what is missing from that list: the model. Governance does not depend on how clever or well prompted the agent is. It sits between the agent and the system it wants to change, and it treats the agent as an untrusted requester, whatever model runs behind it.

TrustRail is a public design study. Its README describes a V1 working against synthetic sandboxes (a fictional merchant and a fictional SaaS product), and states that no real money moves. It is not a production system, and I do not present it as one. It is a way to make the ideas concrete and testable.

What does a policy layer solve?

A policy layer replaces “the prompt says it should not” with a rule that is evaluated before each action and cannot be argued with. A prompt is a request to the model. A policy is a decision taken outside the model.

Three design choices matter more than the rules themselves. First, the policy engine should be pure: given a request, a grant and a set of rules, it returns a verdict, with no database and no side effects, so it is easy to test exhaustively. TrustRail describes its policy engine in exactly those terms, with grant chain validation alongside it.

Second, authority should be delegated explicitly. An agent acts on behalf of a person or a company, and the grant that says so should be checked on every call, not assumed. Third, the verdict should be one of three outcomes, not two. Permitted and blocked are obvious. The third, held for approval, is what makes governance usable instead of obstructive.

When should a human approve an agent action?

Not always. If every action waits for a person, the agent is a slow form and people start approving without reading. If nothing waits, you have given an unexplained actor write access. The answer is to tie the control to the consequence of the action.

The table below is my own framing, not a TrustRail specification. The example actions are the kind the TrustRail README mentions (refunds, credits, plan changes, invitations); the tiers and controls are what I would start from.

• Put the limit in the policy, not in the agent’s instructions.
• Show the approver the request, the grant and the reason it was held, so approval is a decision and not a click.
• Treat a blocked attempt as information: it tells you where the agent’s behaviour and its mandate disagree.

What is a signed receipt, and why is it more than a log?

A log is written by the system that did the thing, and whoever controls that system can change it. A signed receipt is a record that a third party can verify without trusting the system that produced it.

TrustRail’s README describes receipts signed with Ed25519 (as JWS) and a separate receipt verifier package, and says a flow was verified independently. That separation is the point. If the check needs the same database that holds the data, it proves little. If anyone with the public key can check a receipt, it can be handed to an auditor, a customer or a colleague as evidence.

The receipt is also where the five governance questions end up: the requesting agent, the authority, the limits, the verdict and the outcome, in one verifiable record. It turns “we think the agent did this” into “here is the evidence”.

How should an agent earn its autonomy?

Autonomy should be a setting that the evidence moves, not a decision made once at launch. I use the same logic in NOVA, where the security and governance layer includes an autonomy policy and external actions always follow the company policy. NOVA is used by 800 employees at Devoteam, and that scale is exactly why “the agent can do anything the tool allows” is not an acceptable default.

My working sequence, which is my own method rather than a standard:

• Start with read-only and with every consequential action held for approval.
• Watch what the approvers do. Actions they approve unchanged, every time, are candidates for a limit-based rule.
• Widen the grant in small steps, with explicit limits, and keep the ability to withdraw it.
• Narrow it straight away when receipts, errors or incidents show something you did not expect.

What should you log for every agent action?

Log enough to reconstruct the decision, not only the outcome. A line saying “refund succeeded” is not an audit trail. These are the elements I would require on every protected action:

• Who: the agent identity and the person or company that authorised it.
• What: the requested operation and its parameters.
• Under which rule: the grant and the policy version evaluated.
• The verdict: permitted, blocked or held, and the reason.
• Who approved, when, and what they saw, if a human was involved.
• The result in the target system, with an idempotency key so a retry does not repeat the action.
• The signature, so the record can be verified later.

How far can you trust a design study like TrustRail?

Less than a product, and that is the honest answer. TrustRail’s README points to a build status document and a threat model that describes what it does and does not protect against, and says its test report records commands actually run, not claims. I find that tone more useful than a feature list, and it is the tone I try to keep here.

What it offers today is concrete: TypeScript and Python SDKs, an MCP adapter exposing get_order, request_refund and get_action_status, runnable example agents, and two synthetic use cases. What it does not offer is evidence from real customers, real money or a regulated environment. Keeping audit trails is also a sensible reason to build this way in general, since rules such as the EU AI Act push organisations towards traceability, but a design study is not a compliance answer.