Engineering notes
AI safety 12 min read

Human approval for AI agent actions in a multi-tenant SaaS

Covers

AI agents, approval policies, and server-side enforcement

Let the agent propose the action, then put a named human decision between that proposal and the side effect that changes customer data.

Gate the side effect, not the model’s reasoning

An agent can draft a refund, propose an access change, or prepare a customer message without immediately performing it. The clean control point is the moment a proposal would become a real side effect: money moves, data changes, a message leaves the system, or a privileged tool runs.

That boundary gives everyone a clear contract. The agent proposes. A policy names the people who may decide. A reviewer sees the exact action and its context. Your server checks the result immediately before execution.

The approval path

Agent

Propose

Store the exact intended action

incld

Request

Match policy and notify reviewers

Human

Decide

Review context and approve or reject

Your server

Enforce

Check approval, then execute

Use approval selectively. A read-only search usually does not need the same ceremony as a refund or a production deployment. Route high-impact actions through a small set of explicit verbs such as refund, publish, or grant_access.

Create an immutable proposal before asking for approval

Save the exact proposed operation in your own database first. Give it a stable ID, organization ID, agent identity, typed payload, explanation, creation time, and expiry. The record should be immutable once it enters review.

That matters because a reviewer must approve one specific action. If the agent changes the amount, recipient, environment, or permissions, create a new proposal and a new approval request. Never let an approval for one payload authorize a later variation.

Store on the proposal Why the reviewer and executor need it
organizationIdKeeps the action inside the customer boundary
agentId and model run referenceAttributes where the proposal came from
typed action and payloadDefines exactly what execution will do
human-readable explanationMakes the decision understandable without reading a trace
status and expiryPrevents stale proposals from lingering indefinitely
execution idempotency keyMakes retrying the approved action safe

Define who can approve before the agent asks

incld approval policies are scoped to an organization and match the request’s resource ID. A pattern such as agent-action:* covers this class of proposals while keeping the protected action explicit on each request.

Create a tenant-scoped approval policy ts
import { Incld } from "@incld/client"

const tenantIncld = new Incld({
  apiKey: process.env.INCLD_SECRET_KEY!,
  scope: {
    organizationId: session.organizationId,
    userId: session.user.id,
  },
})

await tenantIncld.approvalPolicies.create({
  resourcePattern: "agent-action:*",
  allowedApprovers: ["user_ops_1", "user_security_1", "user_finance_1"],
  mode: "quorum",
  requiredApprovals: 2,
  allowSelfApproval: false,
})

Choose any for one valid reviewer, all when every named reviewer must agree, or quorum for a threshold. Disabling self-approval prevents the proposing agent identity from recording a decision.

The matching policy is snapshotted onto each approval request. Reviewers can therefore see the rule that governed the decision even after a policy is updated for future requests.

Request approval with the same identity you will check

Construct the incld client with the customer organization and a stable service actor for the agent. Use the proposal ID in both the resource ID and the idempotency key, so a retry returns the same logical request instead of opening a second review.

Request approval for one proposed action ts
const agentIncld = new Incld({
  apiKey: process.env.INCLD_SECRET_KEY!,
  scope: {
    organizationId: proposal.organizationId,
    userId: "agent:billing-assistant",
  },
})

const approval = await agentIncld.approvals.create(
  {
    resourceType: "agent_action",
    resourceId: `agent-action:${proposal.id}`,
    action: "execute",
    title: `Refund ${proposal.currency} ${proposal.amount}`,
    description: proposal.explanation,
    metadata: {
      customerReference: proposal.customerReference,
      riskLevel: proposal.riskLevel,
    },
    expiresAt: proposal.expiresAt,
  },
  { idempotencyKey: `agent-action:${proposal.id}:approval` },
)

The resource tuple—resourceType, resourceId, and action—is the join between the proposal, the human decision, and the final server check. Treat it as a stable contract.

Give reviewers the context needed to make a real decision

A queue is only useful when a reviewer can understand the consequence quickly. Put the proposed amount, destination, environment, affected customer, agent explanation, and expiry near the approve and reject actions. Link back to the underlying record when deeper investigation is useful.

A reviewer queue with actions and timeline tsx
"use client"

import { useState } from "react"
import { ApprovalDetails, ApprovalInbox } from "@incld/react-approvals"

export function AgentApprovalQueue() {
  const [selected, setSelected] = useState<string>()

  return (
    <div className="grid gap-6 lg:grid-cols-[22rem_1fr]">
      <ApprovalInbox
        view="assigned"
        filters={{ status: "pending" }}
        onSelect={approval => setSelected(approval.id)}
      />

      {selected && (
        <ApprovalDetails
          approvalId={selected}
          showActions
          showTimeline
        />
      )}
    </div>
  )
}

Rejection should be a first-class outcome, not an error path. A clear reason gives the operator a useful record and gives your application enough context to close the proposal or let the agent prepare a materially different one.

Check approval on the server immediately before execution

Reviewer UI improves the experience, but it cannot authorize a protected mutation. The browser can be stale or bypassed. Load the immutable proposal on your server, rebuild the same tenant and service-actor scope, check the exact resource tuple, and only then perform the side effect.

Enforce the approval at the side-effect boundary ts
export async function executeApprovedAgentAction(proposalId: string) {
  const proposal = await proposedActions.getImmutable(proposalId)

  const agentIncld = new Incld({
    apiKey: process.env.INCLD_SECRET_KEY!,
    scope: {
      organizationId: proposal.organizationId,
      userId: "agent:billing-assistant",
    },
  })

  const state = await agentIncld.approvals.check({
    resourceType: "agent_action",
    resourceId: `agent-action:${proposal.id}`,
    action: "execute",
  })

  if (!state.approved) {
    throw new Error(`Agent action is ${state.status}`)
  }

  return refunds.create({
    amount: proposal.amount,
    currency: proposal.currency,
    customerId: proposal.customerId,
    idempotencyKey: `agent-action:${proposal.id}`,
  })
}

Keep the check close to execution. If the operation enters a queue, pass the proposal ID rather than a browser-supplied payload, then repeat the check in the worker before calling the external system. Use the proposal’s idempotency key for the domain operation so a worker retry does not repeat the effect.

Keep the proposal and decision trail together

incld records the policy snapshot, lifecycle events, and attributed decisions for the approval. Your application should link that approval ID to the immutable proposal and record the eventual execution result beside it.

Question Where to answer it
What did the agent propose?Your immutable proposal record
Which policy and reviewers applied?The incld policy snapshot
Who approved or rejected it, and why?incld decisions and timeline
What actually happened?Your domain execution record and external receipt
Which customer did it belong to?The enforced organization scope on both systems

This split keeps each system authoritative for what it knows: incld for the human decision lifecycle, and your application for the proposed and executed business operation.

A compact production checklist

  1. 1 Only high-impact, explicitly named agent actions enter the approval path.
  2. 2 The exact proposed payload is stored immutably before review begins.
  3. 3 Organization and actor identity come from trusted server context.
  4. 4 Policies name real reviewers, use an intentional resolution mode, and disable self-approval where separation of duties matters.
  5. 5 Approval creation has a stable idempotency key tied to the proposal.
  6. 6 Reviewers see the consequence, affected customer, agent explanation, and expiry.
  7. 7 The server checks the exact resource tuple immediately before the side effect.
  8. 8 Domain execution uses its own idempotency key and records the outcome.
  9. 9 The proposal, approval ID, decision history, and execution receipt can be traced together.

The core design is deliberately simple: the agent may suggest; the human may decide; the server alone may act. That separation makes an approval meaningful instead of decorative.

Put one agent action behind a human decision.

incld provides tenant-scoped policies, reviewer UI, notifications, and durable decision history. Your server remains the enforcement point.