Taking AI Agents to Production: Permissions, Human Approval and a Rollback Plan

An AI agent giving the right answer in a demo does not mean it will work reliably in a live system. The real question when moving to production is: when the agent misunderstands, what actions can it take, and who can stop them? This article proposes a pilot design that technical teams and process owners can use together.

First, narrow the task and the success condition

Instead of a broad goal like “automate customer operations”, choose a well-bounded job such as “classify the incoming support ticket, find the relevant knowledge base entry and prepare a reply draft”. Make sending the draft a separate stage. Anthropic distinguishes between workflows that follow predefined steps and agents that dynamically choose how to use tools, and recommends that additional complexity deliver measurable benefit. Source for the architectural approach.

Define permissions in three layers

• Data: Which records can it read? Could another customer’s record get mixed into the results?
• Action: Are creating a draft, updating a record and sending separate tools?
• Impact: How many people or records could one wrong action affect?

OWASP treats excessive functionality, permissions and autonomy as separate sources of risk. Recommended mitigations include enforcing authorization in the application and API layer rather than leaving it to the model’s judgment, and requiring human approval for high-impact actions. OWASP Excessive Agency.

Show the concrete outcome on the approval screen

A “Continue?” prompt on its own is not enough. In the design we recommend, the user sees the recipient, the fields that will change, the old and new values and any amount on the same screen. If the user edits the text, the approval is bound to the current version, not the old draft. Tasks that time out or whose scope changes are re-evaluated.

Example pilot: from support ticket to reply draft

1. The agent reads only the ticket assigned to it and the permitted knowledge base.
2. It produces the reply draft, the sources it used and any missing information in separate fields.
3. If there is an ambiguity, such as a missing order number, it asks for clarification instead of taking action.
4. Sending is done by a separate service after review by an authorized employee.
5. Re-running the same request must not send the message a second time; duplicate checks are kept in the application layer.

This scenario is a design example; it is not a completed customer project or a performance result.

Release gate and stop plan

In the pilot, alongside normal tickets, test conflicting instructions, broken tool responses, requests for unauthorized records and timeouts. For each failure, track the outcomes “stopped safely”, “took the wrong action” and “escalated to a human” separately. An overall success rate must not hide a critical permission error.

Before going live, define an owner, action limits, an emergency stop path and a rollback method. Starting with a flow that only produces drafts makes it easier to evaluate the need for broader permissions based on real usage. You can plan your AI-powered business process automation work around these checkpoints.