The demo works. Production doesn't.
You've seen this movie before. Someone wires up an LLM with a handful of tools, gives it a task, and it nails the demo. Then it ships, runs against real inputs for a week, and starts doing weird things: calling the wrong tool, looping forever on a task it can't finish, or quietly producing garbage that nobody catches until a user complains.
The problem usually isn't the model. It's that "agent" got treated as a single loop when it actually needs to be several loops, stacked on top of each other, each one responsible for a different failure mode. Teams building serious agent systems at LangChain, Anthropic, and elsewhere have converged on roughly the same architecture, even though nobody agreed on it in advance. It's worth breaking down, because once you see the layers, a lot of "why is my agent flaky" debugging gets a lot easier.
Layer 1: The basic tool-calling loop
At the bottom is the loop everyone starts with: the model gets a prompt and a set of tools, picks a tool, gets a result back, and repeats until it decides it's done. Frameworks like LangChain's create_agent, OpenAI's Assistants API, or a hand-rolled while-loop around the Anthropic Messages API all implement some version of this.
Picture a documentation bot. It gets a Slack message asking for an update to an API reference page. The loop lets it clone the repo, read the relevant files, edit the markdown, and open a pull request, all without a human typing a single git command. That's genuinely useful, and for low-stakes, easily-reversible tasks, it's often enough on its own.
The catch: nothing in this loop checks whether the output is actually correct. The model decides for itself when it's finished, and it's often wrong about that.
Takeaway: the base loop automates action, not correctness. Treat it as a first draft generator, not a final step, for anything where mistakes are costly.
Layer 2: Verification, or catching your own mistakes
The fix is boring and effective: add a grader. After the agent finishes a pass, run its output through a check, deterministic or LLM-based, and if it fails, feed the failure back into the model and let it try again.
For the docs bot, that grader might run the actual CI pipeline, confirm every link in the diff resolves to a real page, and verify the pull request only touches files related to the original request instead of wandering off and "improving" unrelated sections. None of that requires a person to sit and manually review every PR.
This pattern shows up under different names depending on the stack. LangChain calls it a RubricMiddleware or an after_agent hook. If you're rolling your own harness, it's just: run task, score output against criteria, retry with the score and reasoning appended to context, cap retries at some sane number like 3.
def run_with_verification(agent, task, grader, max_attempts=3):
result = agent.run(task)
for attempt in range(max_attempts):
score, feedback = grader.evaluate(result)
if score.passed:
return result
result = agent.run(task, prior_feedback=feedback)
raise VerificationFailure(f"Failed after {max_attempts} attempts")The trade-off is straightforward: verification costs extra tokens and extra latency per run. For a chatbot answering trivia, that overhead isn't worth it. For anything touching a codebase, a database, or a customer-facing document, it almost always is.
Takeaway: if a wrong answer is expensive to fix later, pay for verification now. Cheap mistakes don't need a grader; expensive ones do.
Layer 3: Wiring the agent into events instead of chat
The first two loops assume someone is sitting there invoking the agent. Real production systems don't work that way. They react to events: a new file lands in a bucket, a cron job fires at 2am, a webhook comes in from Stripe, a message drops in a Slack channel.
This is the layer where an agent stops being a chatbot you poke and becomes infrastructure. The docs bot from earlier isn't triggered by a person opening a chat window; it's subscribed to a Slack channel, and any message posted there kicks off a run automatically. Tools like LangSmith Deployment support cron and webhook triggers directly, and no-code builders like Fleet expose the same idea as "channels" you can point at Slack, email, or a queue.
This is also where reliability engineering habits, retries with backoff, idempotency keys, dead-letter queues, actually start mattering, because now the agent runs unattended and failures need somewhere to go besides a terminal nobody's watching.
Takeaway: an agent that only runs when someone remembers to run it isn't automating anything. Event triggers are what make loops 1 and 2 actually save you time.
Layer 4: Closing the loop on the loop itself
The first three layers get work done. The fourth layer is the one teams skip, and it's arguably the most valuable: using the record of what the agent did to improve the agent itself.
Every run produces a trace: which tools got called, what the grader said, where retries happened, what the final diff looked like. Individually, one trace tells you little. A hundred traces tell you a lot. If the docs bot keeps failing the "scoped diff" check specifically when editing files with YAML frontmatter, that's a signal worth acting on, not noise to ignore.
The hill-climbing loop runs an analysis pass over accumulated traces, looking for repeated failure patterns, and turns those patterns into concrete harness changes: a clarified instruction in the system prompt, a new tool, a stricter grader rubric. LangSmith's Engine is one implementation of this pattern; you can build a cruder version yourself with a scheduled job that pulls failed traces from your logging store and asks an LLM to cluster them by root cause.
For teams running open-weight models, this same trace data becomes training signal for fine-tuning, not just prompt edits. The loop doesn't care whether the "improvement" lands in a prompt, a tool schema, or model weights, it just needs a mechanism to turn accumulated failures into a specific, testable change.
Takeaway: if you're not looking at your traces on a schedule, you're leaving the cheapest source of agent improvement on the table. Log everything, and actually read it.
Where humans still belong
None of this is an argument for removing people from the process. A grader can confirm a link resolves; it can't tell you the tone is wrong for the audience or that a technically-correct change will confuse users. That kind of judgment is exactly what a human reviewer is for, and the four-loop structure gives you clear places to insert one:
| Loop | Where a human fits |
|---|---|
| Agent loop | Approval gate before sensitive tool calls (payments, deletes, prod deploys) |
| Verification loop | Human-as-grader for anything too nuanced for automated rubrics |
| Event loop | Human sign-off before an output reaches an end user |
| Hill-climbing loop | Human review of proposed harness changes before they deploy |
Putting the layers together
| Layer | What it does | Skip it if |
|---|---|---|
| 1. Agent loop | Model calls tools repeatedly until the task looks done | Never, this is the baseline |
| 2. Verification loop | Score the output, retry with feedback on failure | Mistakes are cheap and easily caught by the user |
| 3. Event loop | External events trigger runs automatically | You genuinely only need on-demand, manual runs |
| 4. Hill-climbing loop | Traces get analyzed and fed back into harness improvements | You're pre-launch and don't have production traces yet |
Most teams build layer 1 fast, add layer 2 once something embarrassing ships, add layer 3 once someone gets tired of clicking "run" manually, and never quite get to layer 4. That's the layer worth prioritizing sooner than instinct suggests, because it's the one that compounds. Every other layer makes a single run better; this one makes every future run better.