From prototype to production: an agent hardening checklist

An agent prototype is genuinely easy now. Give a model a handful of tools, a goal and a loop, and it will do something impressive within a day.

Production is a different problem. The agent will be handed inputs nobody anticipated, it will call tools that fail halfway, and it will occasionally decide on a plan that is confidently, expensively wrong. Hardening is the work of making all of that survivable.

Here is the checklist we work through before an agent goes anywhere near real users.

Boundaries

• Least privilege per tool. Each tool gets the narrowest scope that lets it do its job. A read tool cannot write. A write tool touches one table, not the database.
• Explicit irreversibility markers. Every tool is classified: reversible, expensive, or irreversible. Irreversible actions require an approval step, always, regardless of how confident the agent is.
• Input validation at the tool boundary. The agent’s arguments are untrusted input. Validate against a schema and reject early — do not rely on the prompt to keep arguments well-formed.
• Spend and step caps. Hard limits on tokens, tool calls and wall-clock time per task, enforced in code. An agent in a loop is a billing incident.

Failure behaviour

• Every tool can fail. Timeouts, partial results, rate limits, malformed responses. The agent needs a defined behaviour for each — usually retry with backoff, then escalate, never silently continue as if it succeeded.
• A defined stop condition. Both success and give-up need to be explicit states. An agent with no way to say “I cannot complete this” will fabricate completion.
• Idempotency where it matters. If a step may be retried, the underlying action must tolerate being applied twice.
• Graceful degradation. When the model is unavailable, the surrounding product should still function in a reduced mode rather than presenting a broken screen.

Instruction handling

• Untrusted content is data, not instruction. Anything the agent reads — web pages, documents, tickets, emails — may contain text aimed at the agent. It must never be treated as a directive. This is a prompt-injection boundary and it needs testing, not just a sentence in the system prompt.
• Confirmation for anything outward-facing. Sending, publishing, paying, deleting. The user confirms; the agent proposes.
• Provenance in the trace. When the agent takes an action, the log should show what caused it, so an unexpected action can be traced to its source.

Observability

• Step-level tracing. Inputs, outputs, tool calls, timings and cost for every step, retained and searchable. Without this, debugging a failed run is guesswork.
• Trajectory-level evaluation. Not just “was the final answer right” but “did it take a sane path”. A correct answer reached through six wasted expensive calls is a problem waiting to scale.
• Regression suite over recorded trajectories. Replay known-good tasks after every prompt or model change.
• Alerting on rate, not just errors. Track the proportion of runs that hit the step cap, escalate to a human, or end in give-up. A rising give-up rate is the earliest signal that something upstream changed.

Human factors

• Show the plan before the action. For multi-step work, letting the user see the intended sequence catches misunderstandings before they cost anything.
• Make interruption easy. A visible stop control, and a clean state when it is used.
• Be honest in the interface. If the system is uncertain, say so. Users forgive uncertainty; they do not forgive confident wrongness twice.

The short version

Most of this is ordinary engineering discipline — permissions, error handling, logging, tests — applied to a component that happens to be probabilistic. The teams that ship reliable agents are rarely the ones with the cleverest prompts. They are the ones that treated the agent as a distributed system with an unusual failure mode, and engineered accordingly.