An AI agent that works in a demo is not the same thing as one that works in production. The gap between the two is where most agent projects fail — and where most of the real engineering decisions live.
Demos are optimised for the happy path. Production optimises for everything else: the input that does not match the expected format, the API that returns an error, the context window that runs out, the task that cannot be completed because a dependency changed. Building agents that handle all of this reliably requires different decisions than building agents that impress in a Loom recording.
Why Production Agents Fail
Three failure modes account for the vast majority of agent failures in real environments.
Hallucination under ambiguity. Language models are trained to produce fluent, coherent text. When the agent does not have enough information to complete a task correctly, it often produces something that looks correct rather than saying it cannot proceed.
Tool call failures. Agents that interact with external systems depend on those systems behaving predictably. Rate limits, authentication changes, malformed responses, and network timeouts all produce errors that a demo never encounters.
Unbounded cost and latency. An agent that works correctly but makes 40 API calls to complete a task that should take 5 is not viable in production. Token costs compound. Latency compounds. Users lose patience.
Task Decomposition
The first architectural decision that determines whether an agent works in production is how tasks are decomposed.
An agent given a single broad instruction must make many implicit decisions about scope, depth, and format. Those decisions will be inconsistent across runs. An agent given a sequence of explicit sub-tasks produces consistent, predictable output. Each sub-task has a defined input, a defined expected output, and a defined failure condition.
Explicit decomposition also makes debugging possible. When an agent produces wrong output, "which sub-task failed?" is a tractable question. "Why did the agent do the wrong thing?" is not.
Tool Design
Every tool an agent calls should have four properties: a clear name, a clear description, a small and well-defined input schema, and a deterministic output format.
The most reliable pattern is to wrap external APIs in thin adapter functions that validate inputs before the call, handle errors and return structured error objects rather than throwing, and normalise outputs into a format the agent expects. The agent never touches the raw API. It calls the adapter. The adapter handles the messy reality of the external system.
Human-in-the-Loop Gates
The question of when an agent should ask for human approval is an engineering decision that should be made explicitly before the agent is deployed, based on the cost and reversibility of the actions it can take.
A low-stakes, reversible action — drafting an email, updating a CRM field, generating a report — can be automated without approval. A high-stakes or irreversible action — sending an email to a customer, processing a financial transaction, deleting a record — should require approval.
A reliable production agent logs every tool call, costs less than a defined limit per run, identifies when it cannot complete a task and returns a clear failure state rather than a fabricated result — and runs hundreds of times without supervision while surfacing failures clearly rather than silently dropping them.