Stop Shopping for a New Skill — “Loop Engineering” Is the Agent Primitive You Already Had, Run Many Times

Before you buy loop engineering as a greenfield discipline, look at what the loop actually is: the thing an agent already was. Anthropic’s definition has not moved since December 2024 — agents “are typically just LLMs using tools based on environmental feedback in a loop” (Anthropic: Building Effective AI Agents¹). Simon Willison was calling the design of that loop “a critical new skill to develop” nine months before the label caught on. His working definition is the same one: an LLM agent is “something that runs tools in a loop to achieve a goal” (Simon Willison: Designing agentic loops²). The Claude Agent SDK draws the identical cycle as four steps — “gather context -> take action -> verify work -> repeat” (Anthropic: Building agents with the Claude Agent SDK³).

So when Addy Osmani defines the new term — “Loop engineering is replacing yourself as the person who prompts the agent. You design the system that does it instead” (Addy Osmani: Loop Engineering⁴) — read it as a shift in who runs the loop, not the arrival of a new mechanism. The primitive is the same gather/act/verify cycle you have been driving by hand; loop engineering is the decision to stop pressing enter between turns.

The concrete version of “stop pressing enter” is the artifact this whole post keeps returning to: the unattended overnight loop.

# The Ralph loop — a fresh context every iteration, state on disk.
while :; do cat PROMPT.md | agent; done

You point it at a PROMPT.md, walk away in the evening, and read what it built in the morning — the technique Geoffrey Huntley popularized as “Ralph” (Geoffrey Huntley: Ralph Wiggum as a “software engineer”⁵). Nothing in that line is new. It is your single-invocation agent, wrapped in while :; do ... done and handed the keys.

The frame is therefore a migration, not a blank slate: treat loop engineering as your single-invocation habits run many times, not a discipline you learn from zero. That reframe tells you where the breakage lives. It is not in the loop syntax — that is one line. It is in every assumption you baked into the playbook when the unit of work was one invocation and the context window reset the moment the task finished. A loop keeps the seat warm.

This axis is worth naming precisely, because it is easy to confuse with the other one. A loop is one agent iterating sequentially — the same context lineage, turn after turn. Running many agents in parallel is a different problem with different failure modes, the isolation and hand-off concerns I covered in multi-agent context isolation. Loop engineering lives on the sequential axis: not more agents, more iterations of one. Why it matters now is measurable — the length of tasks agents can complete autonomously “with 50% reliability has been doubling approximately every 7 months for the last 6 years” (METR: Measuring AI Ability to Complete Long Tasks⁶). The loops are getting longer, so an assumption that was merely wrong on turn one gets expensive by turn fifty.

Audit Your CLAUDE.md as a Per-Iteration Tax, Not a One-Time Cost — In a Loop It Rides Every Turn and Rots

Re-read every “just in case” line in your CLAUDE.md as a charge you re-pay on every iteration, not a fee you settle once. In a single invocation, a bloated instruction file is a fixed cost. The agent loads it, and either the rule near the bottom gets followed or it falls past the instruction ceiling and gets dropped — silently, once. In a loop, that same file reloads every turn, and each turn stacks its own output on top of it.

An agent running in a loop generates more and more data that could be relevant for the next turn of inference, and this information must be cyclically refined.

Refined is the operative word — the context does not refine itself. Left alone it grows by default, because “each invocation requires a full inference pass, and intermediate results pile up in context whether they’re useful or not” (Anthropic: Introducing advanced tool use on the Claude Developer Platform⁸). The scale that always-on context can reach is easy to underestimate: Anthropic reports having “seen tool definitions consume 134K tokens before optimization” (Anthropic: Advanced tool use⁸). That figure is tool definitions, not your CLAUDE.md — but it is the same shape of problem, an always-on block riding every inference pass whether or not this turn needs it.

Here is the audit, applied line by line to the file your overnight loop is pointed at:

• When a line is architecture the agent can read from the repo → cut it. It is context burn re-paid every iteration for something a grep would surface on demand.
• When a line is a “just in case” preference no real failure ever earned → cut it. In one invocation it was one distractor; across fifty iterations it is fifty.
• When a line is a genuine constraint that must hold → keep it, phrase it as a runnable check, and move it as high in the file as it will go, because position decays.
• When a line is a hard safety boundary → it does not belong in an advisory file at all; a loop with no human at the keyboard is exactly the run where “the agent usually respects this” is not good enough.

The reliability cost of skipping that audit is measured, not asserted. Chroma’s study across 18 models and 194,480 calls found that “models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows” (Chroma: Context Rot⁹). This is not a lone vendor’s marketing line — Chroma coined context rot, and Anthropic documents the same finding on its own product surface: “As token count grows, accuracy and recall degrade, a phenomenon known as context rot” (Anthropic: Context windows¹⁰). A single invocation pays context rot once; a loop pays it on every turn, and the bill compounds with the output the loop keeps adding.

Presentation Over Presence: Cut for How the File Reads on Turn 50, Not Whether It’s Present on Turn 1

The instinct when a loop misbehaves is to add a line — spell out the rule the agent missed. That optimizes for presence: the rule is now technically in the file. But presence is the wrong target. Chroma’s sharper finding is that “whether relevant information is present in a model’s context is not all that matters; what matters more is how that information is presented” (Chroma: Context Rot⁹). On turn one, a 300-line CLAUDE.md and a 60-line one both technically contain your rule. By turn fifty, with the loop’s own output layered on top, the 60-line file is the one where the rule is still legible. Cut and order for how the file reads deep into the run, not for whether the rule is present at the start.

Read a Clean One-Shot Demo as the Weakest Evidence You Have — a Loop Is a Different Reliability Regime

Author’s judgment. That a loop fails quieter than the demo — not just more often — is my inference, not a measured result. It follows from two sourced premises: models self-condition on their own prior errors (Sinha et al.), and autonomous runs carry “the potential for compounding errors” (Anthropic). The clean first turn is the least informative data point you have about the fiftieth.

A flawless single run is the worst evidence you can bring to a decision about autonomy, because per-step reliability is a different quantity from single-turn reliability, and the gap is measured.

the per-step accuracy of models degrades as the number of steps increases. This is not just due to long-context limitations — curiously, we observe a self-conditioning effect — models become more likely to make mistakes when the context contains their errors from prior turns.

Their sharpest data point cuts directly against extrapolating from a clean demo: “larger models can correctly execute significantly more turns even when small models have near-perfect single-turn accuracy.” Single-turn accuracy does not predict how many turns a model survives.

Carry that result with its caveats, because the honest version is more useful than the scary one. It was measured on a synthetic running-sum task, not a literal tool loop; the same authors find “thinking mitigates self-conditioning”; and, counterintuitively, larger models are more prone to self-conditioning, not less. So the claim is not “big models get dumb in loops.” It is narrower and harder to dodge: single-shot accuracy doesn’t predict loop reliability, and the mechanism that breaks the extrapolation is the model reading its own earlier mistakes as ground truth.

Here is that mechanism on the running artifact. The overnight loop makes a bad call at iteration 12 — say it edits auth.go against a misread of the token-refresh flow and commits it. Every iteration after 12 opens a fresh context, reads the repo as it now stands, and finds that wrong auth.go sitting there as established fact. The loop is no longer debugging its mistake; it is building on it. By 3 a.m. it has spent thirty iterations making the codebase consistent with a decision it never should have made — and the terminal shows steady, confident progress the whole time. That is the compounding-errors path Anthropic warns about (Anthropic: Building Effective AI Agents¹), running unattended. The blast radius of one bad call is no longer a single turn you can catch; it is every turn that reads the commit.

# One bad call compounding — each fresh context reads it as fact, not mistake.
iter 12    misreads the token-refresh flow, edits auth.go, and commits it
iter 13    fresh context reads the wrong auth.go as established fact
  ...      builds on the bad commit instead of debugging it
+30 iters  3 a.m.: the repo is consistent with a call it never should have
           made — and the terminal showed steady, confident progress throughout

This is why the demo lies by omission. Geoffrey Huntley, who popularized the overnight loop, is blunt about its nondeterminism: Ralph goes off track, and you can wake up to a broken codebase that doesn’t compile unless you build in programmatic verification rather than letting the model grade its own work (Geoffrey Huntley: Ralph Wiggum as a “software engineer”⁵). The demo you were shown is the run that shipped clean on the first try and got kept. The runs that went off track do not get recorded — which is precisely why a clean one-shot demo is the weakest evidence in the room.

Budget the Two Fields a Single Prompt Never Had: An Explicit Stop and Externalized State

A single prompt has two things a loop does not: a natural end, and a memory that lasts exactly as long as the task. The task finished, the window cleared, and “done” handled both. A loop inherits neither for free — so before you walk away, write a stop spec and a state file.

Start with the stop. Anthropic’s baseline is that “the task often terminates upon completion, but it’s also common to include stopping conditions (such as a maximum number of iterations) to maintain control” (Anthropic: Building Effective AI Agents¹). The field guide that covers this most concretely names three, and an unattended loop needs all of them:

1. Max iteration count — a hard ceiling on turns, full stop. Your while :; do ... done has no upper bound; give it one.
2. No-progress detection — if a pass produces no measurable change against the check, halt instead of grinding.
3. A token or dollar budget — a hard spend ceiling that ends the run (Truong Phung: The Agentic Loop / Loop Engineering: A Practical Field Guide¹²).

The hardened version of the running artifact is only a few lines longer, and every added line is a brake:

# The overnight loop, with the two fields a single prompt never needed.
for i in $(seq 1 "$MAX_ITERS"); do          # (1) max-iteration ceiling
  cat PROMPT.md progress.txt | agent        # externalized state, reloaded each turn
  ./run_checks.sh || true                   # the verification the model doesn't self-grade
  git add -A
  git commit -m "iter $i" || break          # (2) nothing to commit -> no progress -> stop
done
# (3) the spend ceiling lives on the wrapper: a wall-clock or token budget that kills the run.

The cap makes the run bounded; the failed commit makes each pass verifiable — nothing changed against the check, so the loop stops instead of grinding on. That is what turns an open-ended loop into bounded, verifiable work.

Then the state. Anthropic’s harness work names it exactly: “the core challenge of long-running agents is that they must work in discrete sessions, and each new session begins with no memory of what came before” (Anthropic: Effective harnesses for long-running agents¹³). Their conclusion is the one that breaks the single-prompt playbook: “even a frontier coding model like Opus 4.5… will fall short of building a production-quality web app if it’s only given a high-level prompt.” The fix is not a better prompt; it is state on disk — Anthropic reconstructs it “with the claude-progress.txt file alongside the git history,” and the field guide reduces it to a line worth taping to your monitor: “the agent forgets; the repo doesn’t” (Truong Phung¹²).

Carry one version caveat so you don’t over-learn the memoryless framing. On newer models the strict “no memory of what came before” is softer than it sounds. Server-side compaction and the Agent SDK’s “compact feature automatically summarizes previous messages when the context limit approaches” (Anthropic: Building agents with the Claude Agent SDK³) both blunt the hard session boundary. The mechanism is version-dependent; the discipline is not. Externalized state is what survives a window reset regardless of how gracefully your model compacts, so write the progress.txt either way.

State also changes what retrieval means, and this is where the loop reopens a problem you thought you had scoped. Just-in-time retrieval, in a single invocation, is one agent resolving pointers at runtime — and it has its own quiet failure modes (where JIT retrieval silently breaks). A loop adds a second retrieval axis with no single-shot analog: pulling the right slice of its own past sessions back into a fresh window. Anthropic’s managed-agent architecture makes that axis explicit: the session is “a context object that lives outside Claude’s context window.” A getEvents() interface “allows the brain to interrogate context by selecting positional slices of the event stream” to pick up where it last stopped (Anthropic: Scaling Managed Agents¹⁴). That is cross-iteration retrieval, and it is its own discipline. Keep the run’s progress log separate from your durable skills and rules, because they answer different questions: what did this run do, versus what should every run know.

Before You Press Go, Run Each Single-Shot Assumption Through Its Loop-Equivalent Check

Do not scale autonomy on a playbook you have not re-derived for many invocations. Every assumption in the table below was safe when the unit of work was one invocation; each one changes when the unit becomes many. Before the overnight loop runs unattended, walk each row and confirm you have an answer — this is the migration made into a checklist.

The through-line is one sentence. A loop is many invocations, not one — so the question before you press go is not “is my prompt good?” but “does every assumption I tuned for a single invocation still hold on turn fifty?” The single-invocation playbook was not wrong; it was scoped. Loop engineering is what you get when you re-derive it for the unit of work you actually shipped — and the reader who runs that migration keeps the overnight wins instead of the overnight surprises.

I write one long-form piece at a time on context engineering and agent reliability. If re-deriving the playbook every time the unit of work changes isn’t how you want to spend the quarter, subscribe and the next one lands in your inbox.

References

Research and Data

1. Chroma: Context Rot — How Increasing Input Tokens Impacts LLM Performance (Hong, Troynikov, Huber)⁹ — Across 18 models and 194,480 calls, performance grows increasingly unreliable as input length grows, and presentation matters more than presence. Backs the per-iteration rot and “presentation over presence” sections.
2. Sinha, Arun, Goel, Staab, Geiping: The Illusion of Diminishing Returns — Measuring Long Horizon Execution in LLMs¹¹ — Per-step accuracy decays and models self-condition on their own prior errors; larger models are more prone and thinking mitigates it. Measured on a synthetic running-sum task. Backs “a loop is a different reliability regime.”
3. METR: Measuring AI Ability to Complete Long Tasks⁶ — Task length agents complete autonomously with 50% reliability has been doubling roughly every 7 months. The “why now” for taking loops seriously.

Practitioner Guidance

4. Anthropic: Building Effective AI Agents¹ — An agent is “just LLMs using tools … in a loop”; the compounding-errors warning; stopping conditions such as a maximum number of iterations. Backs loop-as-primitive and the stop spec.
5. Simon Willison: Designing agentic loops² — Designing the loop is “a critical new skill,” and an agent “runs tools in a loop to achieve a goal” — the term’s lineage predating its coinage. Backs loop-as-primitive.
6. Addy Osmani: Loop Engineering⁴ — Defines loop engineering as replacing yourself as the person who prompts the agent. The term the post reframes.
7. Anthropic: Effective context engineering for AI agents⁷ — A looping agent generates more and more data that must be cyclically refined; names context rot and attributes it to Chroma. Backs “the context accumulates every turn.”
8. Anthropic: Context windows¹⁰ — Accuracy and recall degrade as token count grows (context rot), so curation matters as much as capacity. Official-docs corroboration.
9. Anthropic: Introducing advanced tool use on the Claude Developer Platform⁸ — Intermediate results pile up in context whether useful or not; tool definitions have consumed 134K tokens before optimization. The 134K figure is tool definitions, used only as an analog for always-on bloat.
10. Anthropic: Effective harnesses for long-running agents¹³ — Long-running agents work in discrete sessions with no memory of what came before; a high-level prompt alone fails; state reconstructed from a claude-progress.txt file alongside git history. Backs externalized state.
11. Anthropic: Scaling Managed Agents¹⁴ — The session is a context object that lives outside the context window; getEvents() selects positional slices of the event stream. Backs cross-iteration retrieval as its own discipline.
12. Anthropic: Building agents with the Claude Agent SDK³ — The gather/act/verify/repeat cycle and the compact feature that summarizes previous messages as the window fills. Backs loop-as-primitive and the compaction caveat.
13. Geoffrey Huntley: Ralph Wiggum as a “software engineer”⁵ — The unattended overnight loop with a fresh context each iteration and state in files; verify success programmatically rather than trusting the model to self-assess. The running artifact and the honest failure mode.
14. Truong Phung (DEV Community): The Agentic Loop / Loop Engineering: A Practical Field Guide¹² — The three hard stops (max iterations, no-progress detection, spend budget) and disk-backed state: “the agent forgets; the repo doesn’t.” Backs the stop-and-state section.

Author’s Judgment (not directly sourced)

The following claim is my own synthesis. It follows logically from the sourced material above, but no source states it directly:

• “A loop fails quieter than the demo, not just more often” — Inferred from Sinha et al.’s self-conditioning result (the model reads its own earlier errors as ground truth) and Anthropic’s compounding-errors warning. Neither source states the “less visibly” framing; that is my inference.

Distrust the Dashboard — the Aggregate Number Is a Lagging Indicator, Not a Control

Stop reading the cost dashboard as a guardrail. It is a reconciled report, and on Anthropic’s Enterprise Analytics the numbers for a given day keep moving for a month as reconciliation catches up:

“Values for a given date can be revised for up to 30 days as late events arrive and reconciliation runs. For invoicing-grade totals, query dates at least 30 days in the past.”

Thirty days is the invoicing-grade horizon, not the operational one. A number that is only trustworthy after a month is a report you close your books against — it is not a signal you throttle spend on. The provider’s own reporting layer is built for reconciliation, which means it is a lagging indicator by construction, and a lagging indicator can only ever tell you about damage that already landed.

The granularity is wrong too. Anthropic’s cost endpoints report “per-user and organization-level token usage and cost over time (usage-based Enterprise plans)” (Anthropic: Analytics APIs¹) — per user and per org, never per request. So even once the number stops moving, it cannot tell you which task burned the money.

The failure mode this produces is well-documented, and it is not subtle. A $38k AWS Bedrock bill from a single prompt-caching miss failed silently because the account’s alerts were never a control in the first place:

“Budget alerts are not a kill switch. Credits are not protection.”

That is the take in one line. An alert fires after the invoice; a kill switch fires before the next call. Your actual control has to live upstream of the number, because the number is the thing that arrives too late. Everything that follows is about building the object that upstream control keys on — the task.

Commit the (Agent, Task, User, Team) Dimensions on Every Call — as a Schema, Before You Ship

Decide your attribution dimensions at design time and stamp them on every agent call. A tag you bolt on downstream lives on the wrong object and never reaches the API call — this is the structural failure that makes cloud FinOps tooling useless on LLM spend:

“Tags exist in AWS/GCP/Azure. They don’t propagate to OpenAI/Anthropic API calls. The tag lives on the EC2 instance making the API call, not on the API call itself.”

The tag on the EC2 instance can’t see inside the token bill. Attribution has to be a schema on the call, decided before you ship — because you cannot retroactively stamp a dimension onto calls that already went out untagged.

The good news is that the schema is a configuration decision, not a build. The dimensions you need already exist as telemetry attributes you can emit today. When you see a dimension you want to attribute on, map it to the attribute that already carries it:

• The “what” — which agent, skill, or plugin spent. Emit agent.name, skill.name, and plugin.name. Anthropic’s monitoring describes exactly this: “Attributing spend to specific skills, plugins, or subagent types via the skill.name, plugin.name, and agent.name attributes” (Anthropic: Monitoring⁴).
• The “who” — which person or team. Emit user.email: “When authenticated via OAuth, user.email is included in telemetry attributes” (Anthropic: Monitoring⁴). Team rolls up from there.
• The “which task” — the unit spend actually maps to. Emit prompt.id (the next section is entirely about why this one is the load-bearing dimension).

Each row is a dimension mapped to an attribute you can already emit — so committing the schema is a decision, not an engineering project. The one cost to respect is cardinality: every custom dimension becomes a label on every metric series, so pick the dimensions a ceiling will actually key on and stop. A well-specified task with a defined “done” — the kind the anatomy of a perfect agent task argues for — is what makes these dimensions worth stamping. A bounded task is the thing a per-session ceiling can later be scoped to. Get the schema right here and every section after this one has an object to attach to.

Attribute to the Prompt, Not the User — the Task Is the Unit That Maps to Spend

Make the prompt your attribution key, not the user. One prompt fans out into many billed calls, and user granularity hides which task caused the spend — so the key you attribute on has to be the task itself. Anthropic gives you exactly that key: prompt.id.

Take one concrete agent run. A developer submits a single prompt — “add phone number support to user registration” — and that one prompt does not make one API call. It expands. Anthropic’s own correlation key is what ties the expansion back together:

“To trace all activity triggered by a single prompt, filter your events by a specific prompt.id value. This returns the user_prompt event, any api_request events, and any tool_result events that occurred while processing that prompt.”

Read that fanout carefully, because it is the whole argument. One prompt.id returns one user_prompt event, many api_request events, and many tool_result events. The task is one; the billed calls are many. If you attribute at the user level, all of that collapses into a single “this developer spent $4 today” number that cannot tell you the $4 came from one runaway task rather than forty healthy ones. Attribute on prompt.id and the task is the row — the fanout stays underneath its own key.

This is the part the FinOps field agrees is the actual hard problem — not measuring spend, but mapping it back to who caused it:

“More acute are the challenges of identifying the consumer of the model output, which is especially difficult when the consumers of the same model can be different interfaces/functional modules in the same user application”

The consumer of the model output is the task, not the person. prompt.id is the dimension that names it. Carry that key — one prompt, its prompt.id, and the call fanout underneath — through the rest of this post; it is the object every remaining move operates on.

Expect Per-Task Cost to Keep Climbing — Autonomy Compounds Calls per Task

Budget for per-task cost to rise over time, not hold flat. The same prompt.id fanout you just attributed is a moving target: the longer an agent runs autonomously, the more child calls pile up under one prompt. And the length of task an agent can run autonomously is not holding still — it is doubling.

METR measured the trend directly across six years of frontier models:

“The length of tasks (measured by how long they take human professionals) that generalist frontier model agents can complete autonomously with 50% reliability has been doubling approximately every 7 months for the last 6 years.”

The primary paper puts the same result more precisely: “frontier AI time horizon has been doubling approximately every seven months since 2019, though the trend may have accelerated in 2024” (Kwa, West, Becker, et al. (METR): Measuring AI Ability to Complete Long Software Tasks⁷). Every doubling of autonomous run length is more hours of agent working unsupervised, and every extra hour is more api_request and tool_result events under the same prompt.id. The per-task number climbs because the task itself runs longer.

This is also where task sizing feeds back in. The loop that inflates the bill is the same loop that overruns the context window — an under-scoped task is what sends the agent into more exploration, more retries, more calls per prompt. Sizing the task so its diff fits in one sentence, as the task-sizing heuristics argue, is the upstream lever on per-task cost variance; a well-sized task has a smaller, more predictable fanout by construction.

And the cost driver is exactly this fanout, not model price:

“AI cost also scales differently than cloud cost. It moves with prompt size, fanout, retries, and agentic loops.”

Prompt size, fanout, retries, agentic loops — every one of those is a multiplier on calls-per-task, and every one of them trends up as autonomy grows. So a budget you set against last quarter’s per-task cost is already stale. Whatever ceiling you set has to assume the worst case grows, not holds.

Stop Relying on the Provider’s Account Cap — It Caps the Wrong Granularity

Don’t let the provider’s account-level or key-level cap be your guardrail. It can’t tell one runaway session from a hundred healthy ones sharing the same key, so it only fires after the aggregate is already blown — which is after real damage. The problem is not that the cap is weak; it is that it is aimed at the wrong object.

Bad — the account cap as your guardrail. You set a $10k/month cap on the API key. One task enters a retry loop and burns $4k in three days. The key’s aggregate is still under $10k, so nothing fires; the other ninety-nine well-behaved sessions on that key are subsidizing the runaway’s silence. You find out on the reconciled dashboard, thirty days later.

Good — a cap scoped to the object that actually runs away. The runaway is a session, so the ceiling has to be per session, not per key. The provider cap literally cannot make that distinction:

“Provider-level controls operate at the API key or account level, not the individual session level. They cannot distinguish a single runaway session from many well-behaved sessions using the same key.”

That is the wrong-granularity problem stated exactly. An account cap sums across every session, so a single blast-radius task is invisible until it moves the whole account’s total — by which point the money is spent. This is not a tooling gap you can wait out, either. The field has no accepted framework for the harder case:

“Lack of generally accepted frameworks for cost allocation across multi-agent workloads”

No standard framework, no provider control at the session level — which means the enforcement move is yours to build, at the granularity the provider skips. The next two sections are that build.

Enforce by Terminating Before the Next Call — Not Alerting After the Invoice

Put a budget check that fails the request before it is admitted into the call path. The same prompt.id task now hits a pre-admission gate: the check reads current spend and kills the session before the next API call, rather than reporting it after the invoice. This is the difference between a guardrail and a smoke detector.

The mechanism already exists as a proxy pattern. LiteLLM validates spend before admitting the request:

Request  →  budget check (reads current spend from the authoritative store)
             │
             ├─ under budget  →  request admitted, call proceeds
             │
             └─ over budget   →  request FAILS, no call is made

The docs describe it as a hard, pre-admission gate: “every budgeted request validates spend against the authoritative database before being admitted (covering key, team, user, organization, end-user, tag, and per-window budgets)” (LiteLLM: Budgets, Rate Limits¹⁰). The termination is not a warning — it is a failed request: “After the key crosses it’s max_budget, requests fail” (LiteLLM: Budgets, Rate Limits¹⁰).

Note the dimensions the check covers — key, team, user, tag — the same schema you committed two sections ago. The pre-admission check is exactly what that schema was for; without the dimensions stamped on the call, there is nothing for the check to key on. The check terminates the prompt.id task before its next child call fires, instead of a dashboard reporting the task after all its child calls already fired.

Practitioners are explicit that this cut-off — not optimization — is the thing they actually need:

“I needed visibility + limits per agent/task, and the ability to cut it off, not just optimize it.”

Visibility plus limits plus the ability to cut it off. Compaction and smaller models shave cost per step, but they do not stop an agent that is retrying when it shouldn’t. Only a pre-admission check does — because it is the one control that gets to say “no” before the money is spent.

Set the Ceiling per Session — an Unbounded Loop Is Unbounded Spend

Scope the ceiling to the session: a dollar cap and an iteration cap, both keyed on the session id. An agentic loop makes unbounded calls by default, and only a per-session bound turns “unbounded” into a known worst case. The pre-admission check from the last section needs something to check against — that something is a per-session ceiling.

The default state of an agent loop is the problem:

“When agents run agentic loops, they can make unbounded LLM calls, causing unexpected costs.”

Unbounded is the default. You make it bounded with two counters, both scoped to the session your prompt.id task runs in:

# Per-session ceiling — both caps key on the session id
max_iterations:        50      # hard cap on LLM calls per session
max_budget_per_session: 5.00   # dollar cap per session (x-litellm-trace-id)

# On breach: the incrementing counter crosses the cap and the
# next request is refused — the session dies, it does not warn.

LiteLLM exposes exactly these two: Max Iterations, a “Hard cap on the number of LLM calls per session,” and Max Budget Per Session, a “Dollar cap per session (identified by x-litellm-trace-id)” (LiteLLM: Agent Iteration Budgets¹²). The session id is both the attribution key and the enforcement key — the same identity that ties the prompt.id fanout together is what the counters increment on. That is why attribution had to come first: you cannot scope a ceiling to a task you never named.

Two caps, not one, is deliberate. The iteration cap catches a tight loop that makes many cheap calls; the dollar cap catches a few expensive ones. Together they turn “this task could cost anything” into “this task cannot cost more than $5.00 or fifty calls, whichever it hits first” — a bounded, verifiable worst case per task, which is exactly what a per-task budget was supposed to buy you.

The Line You Actually Draw: Attribution Is Table Stakes, Enforcement Is the Decision

Treat attribution as the non-negotiable prerequisite and enforcement as the explicit org decision. Attribution tells you what happened; it does not stop anything. The field names this split exactly:

“Consumption dimensions tell you what was used, not who in your business used it. Allocation is the work of mapping that usage back to teams, budgets, and cost centers.”

Consumption data is table stakes; allocation — and then enforcement on top of it — is the work. So walk each layer and confirm you have both the dimension and the ceiling, not just the dimension:

1. Distrust the dashboard. Confirm your control lives upstream of the reconciled number, not in it.
2. Commit the (agent, task, user, team) dimensions as a schema. Confirm they’re stamped on the call before you ship, not bolted on downstream.
3. Attribute to the prompt, not the user. Confirm prompt.id is your per-task key, so you can name which task to stop.
4. Expect per-task cost to climb. Confirm your ceiling assumes the worst case grows, because autonomy trends up.
5. Don’t rely on the provider’s account cap. Confirm your enforcement is scoped below the account, at the session.
6. Terminate before the next call. Confirm the budget check fails the request pre-admission, not post-invoice.
7. Set the ceiling per session. Confirm both a dollar cap and an iteration cap key on the session id.

The compression, one row per layer above — the attribution move you must have, and the enforcement move that is the actual decision:

Read the table left to right and the thesis falls out: every enforcement move on the right is impossible without the attribution move on its left. You can’t cap what you can’t attribute. Attribution is the prerequisite you must have; the ceiling is the decision you have to actually make — and shipping agents without making it is shipping unbounded spend and calling it a dashboard.

References

Research and Data

1. Anthropic: Analytics APIs¹ — Cost values are revised for up to 30 days and reported per-user/per-org, never per-request; the dashboard is a lagging, reconciled report by construction.
2. METR: Measuring AI Ability to Complete Long Tasks⁶ — Autonomous task length has doubled roughly every 7 months for 6 years, so per-task run length (and call fanout) keeps climbing.
3. Kwa, West, Becker, et al. (METR): Measuring AI Ability to Complete Long Software Tasks⁷ — Primary paper: frontier AI time horizon doubling approximately every seven months since 2019.
4. FinOps Foundation: FinOps for AI Overview⁵ — Identifying the consumer of the model output is the hard allocation problem; no accepted framework exists for multi-agent cost allocation.

Practitioner Guidance

5. Anthropic: Monitoring⁴ — The agent.name/skill.name/plugin.name/user.email attribution attributes and the prompt.id correlation key that ties one prompt’s call fanout together.
6. LiteLLM: Budgets, Rate Limits¹⁰ — Budget checks validate spend before a request is admitted; over-budget requests fail rather than warn.
7. LiteLLM: Agent Iteration Budgets¹² — Agentic loops make unbounded calls by default; Max Iterations and Max Budget Per Session cap them per session id.
8. Scott Castle, CloudZero: Anthropic Shipped An Enterprise Analytics API. We Shipped the Claude Adapter Today.⁸ — AI cost moves with prompt size, fanout, retries, and agentic loops; consumption tells you what was used, allocation is the work.
9. Ravi Kanani, LeanOps Technologies: FinOps for AI Workloads in 2026³ — Cloud tags don’t propagate to the API call; attribution must be a schema on the call itself.
10. Logan Kelly, Waxell: The $400M AI FinOps Gap⁹ — Provider controls operate at the key/account level and can’t isolate a single runaway session.
11. Hacker News: $38k AWS Bedrock bill caused by a simple prompt caching miss² — Budget alerts are not a kill switch; a silent overrun with no hard cap.
12. Hacker News: Ask HN: How are you keeping AI coding agents from burning money?¹¹ — Practitioner demand for per-agent/per-task limits and the ability to cut it off, not just optimize.

Price In The Two Costs JIT Evangelism Drops: It’s Slower, And It’s Only As Good As Your Tooling

Before you adopt just-in-time context, write down the two costs its advocates bury: the pattern is slower than pre-loading, and it is only as good as the tooling underneath it. Anthropic — whose engineering team is the reason most teams reach for JIT in the first place — states both plainly.

“Of course, there’s a trade-off: runtime exploration is slower than retrieving pre-computed data. Not only that, but opinionated and thoughtful engineering is required to ensure that an LLM has the right tools and heuristics for effectively navigating its information landscape.”

This is the tradeoff row that seeds the whole post. JIT keeps the context window nearly empty of payloads and full of pointers — file paths, stored queries, ticket IDs — that the agent resolves at runtime (Anthropic: Effective context engineering for AI agents¹). That is the same pointers-over-pre-loading discipline that makes a good agent task spec work in the first place: hand the agent enough references to pull context just-in-time rather than dumping everything up front (as in the anatomy of a perfect AI agent task). The win is real — a preloaded copy is a snapshot that ages, while a reference resolves to the current state of the file at the moment of use (TrueFoundry: JIT Context²).

But every one of those runtime resolutions is a new place the loop can break, and the default behavior of a broken resolution is not an error — it is a fabrication. So the ledger has two columns: what breaks, and what the agent does instead of erroring. The rest of this post fills it in, one row per failure mode.

The framing is therefore: JIT does not make retrieval free. It makes retrieval late, and late retrieval fails silently unless you engineer the failure to be loud.

Make Every Unresolvable Reference Fail Loud — An Honest Error, Never A Fabricated Payload

A pointer that resolves to nothing is the canonical JIT failure, and left alone it is the worst kind: the model does not see the miss, so it invents the payload. TrueFoundry names the discipline directly — this is the honest column of the ledger.

“an unresolvable reference must surface as an honest error, not a hallucinated payload”

The move is a resolution contract at the boundary. When a pointer resolves, check the outcome before the result re-enters context, and route each outcome to a loud, loop-visible signal:

• When a reference resolves to null or 404 → raise a hard error the loop can catch, not an empty string the model treats as an answer.
• When a token-scoped fetch returns 401/403 → surface auth failure as its own error class, because a token that refreshed an hour ago is exactly the reference that silently rots (Perrone: Why AI Agents Keep Failing in Production³).
• When the resolver times out → return an explicit timeout signal, not a partial payload the model completes on its own.

The reason this matters is mechanical, not stylistic. A dead reference that returns an empty string looks, to the model, like a fetch that succeeded and found nothing worth quoting — so it fills the gap from parametric memory and the answer reads confident. Auth rot is the sharpest version: an agent that worked at 10am is broken by 2pm because a token refreshed, and the automated renewal fails silently (Perrone: Why AI Agents Keep Failing in Production³). Nothing in the loop sees the break unless the resolver is engineered to raise one.

The first ledger row, then, is the hook made concrete:

Treat The Tool Description As The Decision Surface, Not Documentation — Audit It Like Production Code

The reference resolved and raised nothing — good. The next silent failure is upstream of that: the agent picked the wrong tool to resolve with, because the one-line description it read was broken. Tool descriptions are not prose for humans skimming a README; they are the input the model routes on.

“Tool descriptions are not documentation. They are the LLM’s primary decision surface.”

Audit descriptions the way you review production code, because the wrong one routes silently. Here is the same tool, both ways:

Bad: search_records — “Searches records in the database.”

Good: search_customer_records — “Full-text search over the customer table by name, email, or account ID. Returns up to 25 matches. Does NOT cover orders or support tickets — use search_orders / search_tickets for those.”

The bad stub is not a documentation smell; it is a routing hazard. When two tools have overlapping one-liners, the model picks one and never reports the ambiguity — the most common failures are wrong tool selection and incorrect parameters, especially when tools have similar names (Anthropic: Advanced tool use⁵). The scale of the problem is measured: a 2025 study of MCP tool descriptions found 97.1% contain at least one quality issue and more than half (56%) have unclear purpose statements, while augmented descriptions improved task success by 5.85 percentage points (arXiv:2602.14878, reported in Guy / AWS Heroes: MCP Tool Design⁴). Tool invocation, not model reasoning, is the reliability bottleneck: procedural reliability, particularly tool initialization failures, is the primary bottleneck for smaller models (Huang et al.: When Agents Fail to Act⁶).

This adds the second ledger row — the description-quality dependency from the tradeoff, now concrete:

Cut The Catalog, Don’t Just Document It

Auditing each description is necessary but not sufficient — the catalog itself is a cost paid on every step. A bad description hides a good payload; a bloated catalog degrades every selection at once, because tool-selection accuracy climbs as the set shrinks. Defer or search tools instead of loading all of them upfront:

# Bad: every tool definition loaded upfront, re-sent every loop step.
tools = load_all_tools()          # 58 defs, ~55K tokens before message 1

# Good: expose a search tool; defer the long tail until it's needed.
tools = [tool_search_tool] + [
    t for t in load_all_tools() if t.defer_loading is False
]

The upfront-load number is Anthropic’s own: that’s 58 tools consuming approximately 55K tokens before the conversation even starts, and at Anthropic they’ve seen tool definitions consume 134K tokens before optimization (Anthropic: Advanced tool use⁵). Deferring the long tail is not just a token saving — the Speakeasy team’s controlled experiment found that at 107 tools both large and small models failed completely, while at 10 tools performance was perfect (reported in Guy / AWS Heroes: MCP Tool Design⁴). The catalog is context burn with a selection-accuracy penalty stacked on top.

Give Retrieval A Way To Report “Nothing Found” — Semantic Search Never Will On Its Own

Now assume the right tool was selected and the reference resolved. There is still a silent failure hiding inside the retrieval itself: embedding search always returns something, and “something” is what the model fabricates from. A vector search over an answer that does not exist in your corpus does not come back empty — it comes back with the nearest semantic neighbor, ranked and confident.

The fix is to make absence legible. Add a retrieval path that can report no supporting evidence rather than always returning a ranked list — logical or lexical constraints do this where pure embeddings cannot:

• When embedding search returns results but none satisfy an explicit lexical constraint → treat the constraint failure as a “not found” signal, not as license to return the closest semantic match as fact.
• When the top result’s score sits below an evidence threshold → abstain and mark the passage unsupported rather than passing it downstream as a citation.
• When repeated queries keep returning semantically-related-but-off-topic passages → read that as the corpus lacking the answer, which is the legible signal embeddings alone never give you.

The mechanism is measured. Anchoring retrieval in logical queries makes failed searches legible, and legibility is what suppresses the fabrication:

“Repeated failures under explicit lexical constraints provide a clearer signal that required evidence may be absent, whereas Agentic Hybrid may still return semantically related but unsupported passages.”

The effect size is the point: on answer-unavailable questions, the refusal rate rose from 0.767 to 0.828 while the hallucination rate fell from 0.128 to 0.083 (Zeng et al.: Rethinking Agentic RAG⁷). Bad retrieval is not a hallucination in the usual sense — the retrieval worked perfectly, it just retrieved garbage, and there is no mechanism to flag that the result was low-confidence unless you build one (Perrone: Why AI Agents Keep Failing in Production³).

Third ledger row:

Cap The Retrieval Loop And Abstain — With No Stop Rule, “Get More” Becomes A Budget Fire

The prior three failures are about what a single resolution returns. This one is about the loop around them: an agent that can retrieve will, by default, keep retrieving, because a locally-optimizing loop with no stopping rule always answers “do I have enough?” with “get more.”

“At each step, it asks, ‘Do I have enough?’ and when the answer is uncertain, it defaults to ‘get more’. Without hard stopping rules, the default spirals.”

Engineer the stop rule explicitly. The loop does not need to be smart about when to quit — it needs a hard cap and an honest exit:

1. Set a hard cap on retrieval cycles — cap at three passes, not “until confident.”
2. Count a pass as failed when it adds no new supporting evidence, not merely when it errors.
3. At the cap, return a best-effort answer with a confidence disclaimer — abstain from certainty rather than spending another cycle.
4. Deduplicate results across passes so the loop can’t mistake the same low-value chunk for progress.

The mitigation is not novel; the discipline is: three cap retrieval cycles, and after three failed passes return a best-effort answer with a confidence disclaimer (Ibrahim / Towards Data Science: Agentic RAG Failure Modes⁸). The failure mode without it is not theoretical — one team documented agents making 200 LLM calls in 10 minutes, burning $50–$200 before anyone noticed, and another saw costs spike 1,700% during a provider outage as retry logic spiralled out of control (Ibrahim / Towards Data Science: Agentic RAG Failure Modes⁸). A retrieval loop without a cap is a budget fire with no smoke alarm.

Fourth ledger row:

Design The Fallback Once: Cap, Check For Evidence, Abstain Honestly, And Never Dump The Raw Payload Back

Four failures, one fix. Rather than bolting a different guard onto each break, build a single fallback path that every resolution funnels through. Get its last step right — the naive fallback poisons the context it was meant to protect. Run these four in order:

1. Cap the loop. Apply the hard retrieval-cycle cap from the section above before anything else — the fallback runs because a resolution failed, and an uncapped fallback is just the thrash again.
2. Check whether real evidence came back. Distinguish “resolved with supporting evidence” from “resolved with a semantically-related but unsupported passage” — the legibility signal is what tells the two apart.
3. Prefer honest abstention. If no real evidence is present, surface an honest error or a disclaimed best-effort answer — never a confident fabrication. This is the honest column, applied.
4. Refuse to re-inject the raw resolved payload. When a resolution comes back low-confidence, do not dump the raw blob back into context “just in case.”

That last step is where most fallbacks quietly fail. The instinct is to hand the model everything retrieved and let it sort out relevance — but re-injecting a low-value payload rots the very window JIT kept clean. The degradation is measured across 18 models: performance degrades as input length increases, often in surprising and non-uniform ways, and even a single distractor reduces performance relative to the baseline (Chroma: Context Rot — Hong et al., 2025⁹). One irrelevant payload is enough to move the curve — the same context-rot slope that makes task sizing a context problem rather than a lines-of-code problem (as in how to size tasks for AI coding agents). Dumping the raw payload back trades a loud miss for a quiet degradation, which is the worse trade.

The ledger’s second column, rewritten — every “what the agent does instead” turned into “what a hardened agent does”:

The Completed Ledger: Run This Table Before You Ship A JIT Agent

Before you ship a JIT agent, walk the completed ledger row by row — for each break, confirm the agent surfaces an honest error instead of a confident fabrication. Each row maps to one section above; if any row’s fix is unimplemented, that failure is silent in production.

The through-line across all five: JIT does not make retrieval free — it makes retrieval late, and late retrieval fails silently unless you engineer the failure to be loud. The evangelism ships the first column. The honest column is the deliverable. If shipping the fallback for a production JIT agent is the wedge between a demo and a system you can trust, that is the work I do — let’s talk.

References

Research and Data

1. Zeng et al.: Rethinking Agentic RAG — Toward LLM-Driven Logical Retrieval Beyond Embeddings⁷ — Legibility of failed logical search; refusal rate 0.767→0.828 and hallucination 0.128→0.083 on answer-unavailable questions. Backs the “report nothing found” section.
2. Chroma: Context Rot — Hong, Troynikov, Huber, 2025⁹ — 18-model study: performance degrades non-uniformly as input grows, and a single distractor reduces performance. Backs “never dump the raw payload back.”
3. Huang et al.: When Agents Fail to Act — A Diagnostic Framework for Tool Invocation Reliability⁶ — Tool initialization failures are the primary reliability bottleneck for smaller models. Backs the tool-description-as-decision-surface premise.

Practitioner Guidance

4. Anthropic: Effective context engineering for AI agents¹ — The runtime-exploration tradeoff (slower + tooling-dependent) and the pointers-over-pre-loading JIT definition. Backs the opening tradeoff section.
5. TrueFoundry: JIT Context — Why the Best Agents Load Late and Load Little² — The “honest error, not a hallucinated payload” framing and the description-quality dependency. Seeds the ledger and backs the dead-reference section.
6. Anthropic: Introducing advanced tool use on the Claude Developer Platform⁵ — 58 tools/55K tokens upfront, 134K before optimization; wrong tool selection as the most common failure. Backs “cut the catalog.”
7. Guy / AWS Heroes: MCP Tool Design — Why Your AI Agent Is Failing (And How to Fix It)⁴ — Tool descriptions as the LLM’s primary decision surface; the 97.1%/56%/5.85pp figures (arXiv:2602.14878) and the Speakeasy 107-vs-10-tools result. Backs the tool-description section.
8. Ibrahim / Towards Data Science: Agentic RAG Failure Modes⁸ — The “get more” spiral, the three-cycle cap, and the 200-calls/$50–200 and 1,700% cost anecdotes. Backs “cap the loop and abstain.”
9. Perrone: Why AI Agents Keep Failing in Production³ — Silent retrieval failure has no flag; bad retrieval is not a hallucination; auth/token rot fails silently. Backs the dead-reference and fallback sections.

Take one concrete scenario and hold it through the whole post: you are adding a non-trivial feature to a large codebase, and a single agent’s window is filling with codebase exploration, migration spelunking, and unrelated log output before it writes a line. Every gate below is applied to that same feature.

Stop Asking “Is It Parallelizable” — Ask “Is One Window Still Enough”

People reach for a second agent because the work looks splittable. That instinct optimizes for speed, and speed is the wrong trigger. The real trigger is that one agent can only hold so much — large codebases overwhelm a single context window, and once that happens the split is about relieving the window, not racing a clock (Addy Osmani: The Code Agent Orchestra²).

That reframe changes what a multi-agent system is for. In Anthropic’s research setup, the benefit isn’t that many agents run at once — it’s that each one carries its own window and hands back a compressed result:

“Subagents facilitate compression by operating in parallel with their own context windows, exploring different aspects of the question simultaneously before condensing the most important tokens for the lead research agent.”

Read that carefully: the parallelism is incidental, the compression is the point. The extra windows exist so the lead agent never has to hold the raw exploration — it gets the condensed version. That is context isolation doing the work.

Apply it to the feature. The single agent is drowning not because the feature is enormous but because everything it touches — the migration history, the log output, the files it grepped to orient itself — is piling into one window alongside the code it’s supposed to write. This is the same problem sizing solves upstream: a well-sized task is one that stays inside a single window. This post is what to do when the feature is legitimately too big for that, and one window has stopped being enough.

The multi-agent question is therefore: not “can I parallelize this?” but “is one window still enough to hold it cleanly?”

Split When Pollution — Not Size — Breaks the Window

When a single agent’s output degrades, the reflex is to blame length — the context is “too full.” That’s usually the wrong diagnosis. Look for pollution before you look at the token count, because irrelevant content degrades output well before you approach any size limit.

When you see these in the agent’s window, treat it as pollution, not size:

• Search results it grepped once and never reads again — orientation noise that stays resident in context.
• Stale log output pasted in to debug one thing, now sitting there for the rest of the session.
• Dead-end files the agent opened, ruled out, and left in the window.
• Migration spelunking — the history it walked to understand the schema, most of which is irrelevant to the change.

Each of these is a distractor, and distractors are measurably corrosive. Chroma’s context-rot study, which tested 18 models, found the effect starts small and immediately:

“Even a single distractor reduces performance relative to the baseline (needle only).”

The reason this catches people off guard is that the standard benchmarks hide it. Models do well on simple needle-in-a-haystack retrieval, which produces a false sense that long context is handled — “which has led to the perception that long-context is largely solved” (Chroma: Context Rot³). It isn’t. The window fills with distractors and quality erodes, and no token counter warns you, because the count is still comfortably under the limit.

For the feature, this is the actual failure mode. The agent isn’t failing because the change is “big” — it’s failing because the migration spelunking and the log output polluted the window it needed for the code. That’s the same context-rot slope that sizing works to avoid, and the fix is to get the distractors out of the window, not to buy a bigger one.

Isolate the Side-Quest With a Subagent Before You Split the Whole Job

Full multi-agent is the expensive answer. Before you pay for it, run the cheap one: hand the polluting side-quest to a subagent so the main window stays clean. The Claude Code docs name the exact trigger:

“Use one when a side task would flood your main conversation with search results, logs, or file contents you won’t reference again: the subagent does that work in its own context and returns only the summary.”

That is context isolation without a full split. The mechanism is that “each subagent runs in its own context window with a custom system prompt, specific tool access, and independent permissions” (Claude Code Docs: Create custom subagents⁴) — so the exploration happens somewhere else and only its conclusion comes back.

For the feature, the move is to spin the codebase exploration out:

## Subagent: codebase-explorer

### Goal
Locate every call site of the current user-registration flow and the
existing migration pattern. Return a summary, not the raw files.

### Return only
- The 3–5 files the writer agent must edit, with one-line reasons.
- The migration format to mirror (file path + shape), not its full history.
- Any constraint the writer can't infer (naming conventions, ordering).

### Do not
- Paste full file contents or log output back into the main conversation.
- Make edits. This agent reads and reports; it does not write.

The main agent now gets the three-to-five-file answer instead of the grep transcript, the migration shape instead of the migration spelunking. Its window holds the code and the summary — nothing else. This is just-in-time context applied across a window boundary: pull the exploration in as a condensed pointer, not as raw pages.

Reach for a full split only after this stops being enough — when even the isolated summary plus the real work overruns one writer’s window. Most of the time it won’t, and you’ve avoided the 15x bill entirely.

Make the Value Clear the 15x Token Bill Before You Split

If in-session isolation isn’t enough and you’re committing to separate agents, price it first. Multi-agent is not a free upgrade — it is a specific, measurable multiple on cost:

“agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats”

Fifteen times is not a rounding error you can wave through on the assumption that faster is better. And cost isn’t a side concern here — in Anthropic’s own evaluation, “token usage by itself explains 80% of the variance” in performance (Anthropic: How we built our multi-agent research system¹). You’re paying 15x for the same lever that drives the results.

So the test is a value test, not a speed test. The isolation benefit — the clean windows, the compressed hand-back — has to be worth that bill on its own terms. “It would finish sooner” does not clear it; “one window genuinely cannot hold this and the isolation is what makes the output correct” might. If the value isn’t obvious at 15x, that’s your answer: don’t split.

Don’t Split Until Writes Can Stay Single-Threaded

The first gate asks whether one window is enough. The second asks something independent: even if you should isolate context, can you do it safely? The answer turns on writes.

When you see these signals, keep one writer and make the extra agents read-only:

• The task mutates shared state — migrations, code edits, generated files. Parallel writers here corrupt each other.
• Agents would need to hand off partial work — each handoff is a place for their private decisions to conflict.
• The value is in exploration, not action — then the added agents should explore and report, and a single agent should write.

That last pattern is the one that actually works in practice. Cognition’s read on the current state is blunt about which shape survives contact with reality:

“multi-agent systems work best today when writes stay single-threaded and the additional agents contribute intelligence rather than actions”

This is not a fringe constraint — it’s the norm. As Cognition puts it, “most multi-agent setups in the world are limited to ‘readonly’ subagents” (Cognition (Walden Yan): Multi-Agents: What’s Actually Working⁵). The working configuration is read-only intelligence feeding a single writer, not a swarm of parallel writers.

For the feature, that resolves the architecture. The migrations and the code edits are writes — so the safe split is read-only explorer subagents (the codebase-explorer from the last section is exactly this) feeding one agent that does all the writing. You get the context isolation without handing the shared state to a swarm. If you can’t arrange the work that way — if the split genuinely requires two agents writing to the same tree — the second gate fails and you don’t split.

Treat Coordination Readiness as a Test, Not a Vibe

“Can writes stay single-threaded” is the principle; coordination readiness is where you check it against the concrete surface of the actual feature. Don’t decide by optimism. Run the checklist:

1. File ownership — Does each agent own a disjoint set of files, or would two touch the same file? Overlap means a merge conflict you’ll pay for later.
2. Lock-file contention — Would agents run git operations against the same working tree at the same time?
3. Migration ordering — Do the migrations have to apply in a fixed order? Ordered work is not parallel work.
4. Dependency sequencing — Does one piece consume another’s output (the API before the client that calls it)?

Items 2 and 3 are not hypothetical. Concurrent git operations against one working directory fail hard:

“Git uses file-based locking (.git/index.lock, .git/config.lock) to protect repository integrity. When two agents attempt concurrent git operations on the same working directory, the second agent receives a fatal error.”

And ordered work resists parallelization by definition — when one task consumes another’s output, “these tasks need to be sequenced, not parallelized” (Augment Code (Paula Hingel): Git Worktrees for Parallel AI Agent Execution⁶). For the feature, the migration must land before the code that reads the new column, and the lock file means two agents can’t both commit against the working tree. That checklist, run against this specific feature, is what decides whether the split is safe — not a general sense that the work “seems parallel.” A tight spec for each isolated task is what keeps ownership disjoint in the first place.

When the Work Is State-Dependent, Keep It Single-Agent

The checklist has a hard case worth naming as its own rule: when each step mutates state the next step reads, don’t parallelize it. This isn’t a preference — it’s the difference between a large gain and a large loss, measured across 260 controlled configurations:

“Relative performance change compared to single-agent baseline ranges from +80.8% on decomposable financial reasoning to -70.0% on sequential planning, demonstrating that architecture-task alignment determines collaborative success.”

Decomposable work swings up 80.8%; sequential, state-dependent work swings down 70.0%. Whether the split pays off is a property of the task’s structure, not the enthusiasm behind it. Migration-then-code is sequential. Keep it on one writer.

When the Split Backfires, It’s Conflicting Decisions — Not a Weak Model

When a multi-agent run produces bad output, the instinct is to blame the model — “it wasn’t smart enough.” Look at the system design first, because that’s where the failure almost always lives. The mechanism is that agents working in isolation each make small implicit choices, and those choices collide on recombination:

“Actions carry implicit decisions, and conflicting decisions carry bad results.”

One agent names the field phone, another expects phone_number; one assumes the migration ran, another writes as if it hasn’t. Neither is a capability gap — both are coordination failures. The empirical breakdown backs this: a study of multi-agent failures identifies “14 unique modes, clustered into 3 categories: (i) system design issues, (ii) inter-agent misalignment, and (iii) task verification” (Cemri et al. (arXiv): Why Do Multi-Agent LLM Systems Fail?⁹).

Two of those three categories are coordination, and the third is verification — none is “the model is too weak.” That’s why single-threaded writes and the readiness checklist matter more than picking a stronger model: they remove the conflicting-decision surface instead of hoping capability papers over it. When your split backfires, debug the coordination design before you reach for a bigger model.

Cap the Fan-Out at Your Review Throughput

There’s a ceiling on fan-out, and it isn’t set by the model — it’s set by you. More agents generate more output, but generation is no longer the constraint:

“The bottleneck is no longer generation. It’s verification.”

Once verification is the bottleneck, adding agents past what you can review just piles up unreviewed diffs. The practical ceiling is small and empirical: “Three to five teammates is the sweet spot. Token costs scale linearly, and three focused teammates consistently outperform five scattered ones” (Addy Osmani: The Code Agent Orchestra²). The extra fan-out doesn’t buy throughput, it buys review debt.

The human limit is the real bound. As Simon Willison describes his own parallel workflow, “I can only focus on reviewing and landing one significant change at a time, but I’m finding an increasing number of tasks that can still be fired off in parallel without adding too much cognitive overhead to my primary work” (Simon Willison: Embracing the parallel coding agent lifestyle¹⁰). One significant change lands at a time regardless of how many agents ran. Size the fan-out to what you can actually review and land — three to five, not “as many as the model can spawn.”

The Two-Gate Decision: A Flowchart

Both gates have to clear before you split, and they clear in order — isolation first, because if one window is still enough there’s nothing to decide; readiness second, because a real isolation need you can’t coordinate safely is still a no. Run the feature through it end to end:

flowchart TD
    A[Speed wish or isolation need? §1] -->|Just faster| STOP1[Don't split]
    A -->|One window can't hold it| B[Failing from pollution, not size? §2]
    B -->|Distractors: grep noise, stale logs, spelunking| C[Can a subagent isolate the side-quest? §3]
    C -->|Read-only explorer summary fixes it| STOP2[Isolate in-session, stop here]
    C -->|Summary + real work still overruns one window| D[Value clears ~15x token bill? §4]
    D -->|Not obviously worth 15x| STOP3[Don't split]
    D -->|Worth it| E[Writes stay single-threaded? §5]
    E -->|Needs parallel writers to shared state| STOP4[Don't split]
    E -->|Read-only intelligence feeds one writer| F[Coordination readiness passes, task decomposable? §6]
    F -->|Sequential / state-dependent| STOP5[Keep it single-agent]
    F -->|Decomposable, coordination clean| G[Fan-out exceeds review throughput? §8]
    G -->|Yes| CAP[Cap at three to five]
    G -->|No| SPLIT[Fan out]

Gate 0 — Is this a speed wish or an isolation need? (§1) If you want it faster → not a reason to split. If one window genuinely can’t hold it cleanly → continue.

Is the window failing from pollution, not size? (§2) If it’s distractors — grep noise, stale logs, migration spelunking → the target is to get them out of the window, continue.

Can a subagent isolate the side-quest without a full split? (§3) If a read-only explorer returning a summary fixes it → do that, stop here. If even the summary plus the real work overruns one window → continue.

Does the value clear ~15x the token bill? (§4) If the isolation isn’t obviously worth 15x → don’t split. If it is → continue.

Can writes stay single-threaded? (§5) If the split needs parallel writers to shared state → don’t split. If added agents can stay read-only intelligence feeding one writer → continue.

Does coordination readiness pass, and is the task decomposable? (§6) Run the checklist — file ownership, lock-file contention, migration ordering, dependency sequencing. If the work is sequential/state-dependent → keep it single-agent. If it’s genuinely decomposable and coordination is clean → continue.

Would fan-out exceed your review throughput? (§8) If yes → cap at three to five, whatever you can review and land. If no → fan out.

For the feature, the honest walk lands at Gate 3: a read-only codebase-explorer subagent clears the pollution, the main window holds, and you never pay the 15x bill or touch the coordination gate. When a split does clear all gates and still backfires, route the post-mortem through §7 — look for conflicting implicit decisions before you blame the model.

References

Research and Data

1. Anthropic: How we built our multi-agent research system¹ — Subagents isolate context via their own windows and compress findings back; multi-agent uses ~15x the tokens of a chat, and token usage explains 80% of performance variance.
2. Chroma (Hong, Troynikov, Huber): Context Rot³ — Across 18 models, even a single distractor degrades output; NIAH success has falsely convinced people long context is solved.
3. Kim et al. (arXiv): Towards a Science of Scaling Agent Systems⁷ — Across 260 configurations, performance swings +80.8% on decomposable tasks to -70.0% on sequential ones; task structure decides whether the split pays off.
4. Cemri et al. (arXiv): Why Do Multi-Agent LLM Systems Fail?⁹ — Multi-agent failures cluster into 14 modes across three categories (system design, inter-agent misalignment, task verification), not model quality.

Practitioner Guidance

5. Claude Code Docs: Create custom subagents⁴ — Use a subagent when a side task would flood the main conversation; each runs in its own window and returns only the summary.
6. Cognition (Walden Yan): Multi-Agents: What’s Actually Working⁵ — Multi-agent works best when writes stay single-threaded and extra agents contribute intelligence; most real setups are read-only.
7. Cognition (Walden Yan): Don’t Build Multi-Agents⁸ — Actions carry implicit decisions, and conflicting decisions between agents carry bad results.
8. Augment Code (Paula Hingel): Git Worktrees for Parallel AI Agent Execution⁶ — Git’s file-based locking makes concurrent operations on one working tree fail; dependent tasks must be sequenced, not parallelized.
9. Addy Osmani: The Code Agent Orchestra² — Large codebases overwhelm one window; three-to-five agents is the sweet spot, and verification, not generation, is the bottleneck.
10. Simon Willison: Embracing the parallel coding agent lifestyle¹⁰ — You can review and land one significant change at a time, which bounds how far parallel fan-out actually scales.

One concrete scenario runs through every section below: a six-person reviewer pool absorbing the output of a growing agent fleet. The peer-reviewed corpus gives us the exact shape — one team reporting 30 pull requests per day across 6 reviewers (Baltes, Cheong, Treude: “An Endless Stream of AI Slop”¹). The artifact that carries the argument is the ratio: agents-in-flight over reviewer-hours-available — the same six-reviewer team under a different microscope each time.

Your Agents Don’t Hit a Correctness Wall — They Hit a Reviewer-Throughput Wall You Can Already Measure

People instinctively debug the agent. More context, better prompts, a tighter task spec — all aimed at the code the agent produces. But the generation problem is largely solved; the wall you actually hit is downstream, and it’s a throughput wall, not a correctness wall. As Codacy puts it, “The bottleneck has moved from writing code to deciding whether code is safe to merge” (Codacy: AI Is Breaking Code Review²). Generation stopped being scarce; deciding whether the generated code should merge is exactly as expensive as it always was.

The number that proves this is already in your telemetry. Across two years of data on 22,000 developers, the Faros AI Engineering Report 2026 found that median time in review is up 441.5%, and pull requests merged with no review at all — human or agentic — are up 31.3% (Faros telemetry — vendor research, commercial interest; Faros: The Acceleration Whiplash³). That is the six-reviewer team in aggregate: raw agent output climbing while the same six people stay the fixed denominator. Review time doesn’t rise 441.5% because the code got worse — it rises because more of it is arriving at a pool that didn’t grow.

The tell is that agent-assisted output and shippable throughput move in opposite directions. Codacy reports CircleCI’s 2026 data showing feature-branch throughput up 59% year over year while, for the median team, main-branch throughput fell nearly 7% and main-branch success rates dropped to 70.8% (Codacy²). More code enters the pipeline; less of it reaches production. The bottleneck moved — as Developers Digest frames it — “from generation to review queues, CI capacity, flaky environments, branch policy, cost ceilings, and the human attention needed to decide what should actually merge” (Developers Digest: AI Coding Agents Move the Bottleneck to Review Queues⁴).

The asymmetry underneath all of this is the durable part. Addy Osmani lands it in one line:

“We made writing cheap, and understanding stayed exactly as expensive as it has always been.”

So stop tuning the agent and go read your review dashboard. The ceiling on how many agents you can run is reviewer-hours, and the 441.5% is telling you where that ceiling already is.

Do the Reviewer-Capacity Math Before You Approve Agent Number Seven

Adding an agent feels like adding output. It isn’t — it’s adding a queue. Every PR an agent opens is a claim on reviewer-hours you have not yet spent, and if the sum of those claims exceeds the hours your reviewers actually have, the surplus doesn’t ship. It waits. Before you approve agent number seven, do the arithmetic that nobody does: reviewer-hours the new PRs will consume, against reviewer-hours you have.

The six-reviewer team makes the crossover concrete. Hold the denominator fixed — six reviewers, eighteen focused review-hours a day — and watch per-PR attention fall as the fleet grows:

The numbers come from the corpus. At 30 PRs per day (Baltes et al.¹) — five per reviewer — and, generously, three focused review-hours each before meetings and their own work eat the rest, that’s roughly 36 minutes of genuine attention per PR. Agent number seven pushes output to 35 PRs against the same eighteen review-hours, and per-PR attention drops to about 31 minutes. Two more agents and you’re under 27. At some crossover the attention-per-PR falls below the floor where a reviewer can actually reason about the change, and past that point every added agent produces queue depth, not shippable output.

That crossover is not a metaphor. Monperrus, who argues the case as hard as anyone, names it directly: “the review queue becomes the binding constraint on their delivery pipeline” (Martin Monperrus: The End of Code Review⁶). Blake Crosley states the mechanism in one sentence — “An agent-assisted developer produces more pull requests per day than human review capacity can absorb” (Blake Crosley: Agents Supersede the Reviewer, Not the Review⁷). The absorption rate is the fixed quantity. The agent count is the variable you keep turning up against it.

The sizing question is therefore: does the reviewer-hours math clear before you add the agent, or are you buying queue depth and calling it throughput? This is the same discipline as sizing the tasks in the first place — a well-sized diff maps to a reviewable unit, so upstream sizing is what keeps the per-PR review cost inside the budget (How to Size Tasks for AI Coding Agents). Compute the denominator first. If the sum exceeds capacity, agent number seven is not an output gain — it’s a deferral.

Measure Your Review Agents by the Load They Remove, Not the Comments They Post

The reflexive fix for a review queue is a review agent, and the reflexive way to measure it is comment count and merge rate. Both are wrong, and the empirical data says so bluntly. Rip merge rate and comment volume off the review-agent dashboard and track one thing instead: reviewer-load removed.

When you see a review agent’s comment volume climb while its accepted-comment ratio falls, treat it as adding load, not removing it. The accepted-comment ratio is the honest signal — an issue can be considered accepted when a later commit addresses it. Comment count is not; past a threshold, reviewers stop reading. Watch the two together: rising volume plus falling acceptance is a review agent that is now part of the queue it was supposed to drain.

The merge-rate data is the part that should stop you. Chowdhury et al., studying code review agents across thousands of real PRs, found that “CRA-only reviewed PRs achieve a 45.20% merge rate, 23.17 percentage points lower than human-only PRs (68.37%)” (Chowdhury et al.: From Industry Claims to Empirical Reality⁸). High-volume review agents didn’t raise the merge rate — they lowered it by 23 points, which is the exact gap the opening line named. And the reason is noise: in the same study, “60.2% of closed CRA-only PRs fall into the 0–30% signal range” (Chowdhury et al.⁸). Most of the comments were not the signal a reviewer needed.

Two independent lines confirm that comment volume stops mapping to value once it crosses a noise threshold. The Nutanix CR-Bench study finds that “code review agents can exhibit a low signal-to-noise ratio when designed to identify all hidden issues, obscuring true progress and developer productivity” (Pereira et al. / Nutanix: CR-Bench⁹) — a “find everything” agent buries the reviewer. And CodeRabbit, whose commercial interest points the other way, admits the same measurement failure:

“Precision metrics degrade because even high-quality comments may be ignored simply due to volume.”

Once reviewers skim and bulk-dismiss, as Loker notes, you are “no longer measuring how a tool performs in practice, but how reviewers cope with noise.” Load removed — did the reviewer spend fewer minutes, not read more comments — is the only number that tracks whether the review agent bought you capacity or spent it.

Triage Review Depth by Risk Class, Not by Author or Diff Size

Uniform review depth is what exhausts the pool. If every incoming diff earns the same scrutiny — the config bump and the payments change reviewed with identical rigor — you are spending your scarcest resource flat across work that carries wildly different blast radius. The fix is to route by risk class before a human touches the diff.

Bad: Every PR goes into the same review queue at the same depth. A reviewer reads the config change as carefully as the payments path because the process doesn’t distinguish them — so either the config change wastes senior attention, or the payments change gets the same shallow pass everything else gets. Depth is set by whoever is next in the queue, not by what the diff can break.

Good: Tier the diff by blast radius on the way in. Osmani states the rule and the two ends of it exactly:

“Tier by risk, not by author. A config change earns a linter and a glance. A payments path earns the full stack: types, tests, two different AI reviewers, a human who owns that system, and a security pass.”

Same pool, wildly different draw on it — sorted by what the change actually touches rather than who wrote it. The config change spends near-zero reviewer-hours; the payments path spends the full stack.

Risk class is the first axis; diff size is the second. A large diff inside a low-risk boundary is still cheap to review; a small diff on a payments path is not. Keep each agent’s output inside a reviewable unit and the risk-tier draws stay predictable — which is the whole point of sizing the task to comprehension capacity upstream (How to Size Tasks for AI Coding Agents). Triage by risk first, size second, and the six reviewers stop burning full-stack attention on config bumps.

Cap High-Risk AI-Diff Load Per Reviewer and Budget Review as a Finite, Protected Resource

Unbudgeted review load is invisible to your throughput dashboards, and invisible load shows up later as senior-engineer attrition. The capacity math from earlier gives you a per-team number; the move here is a per-person budget you then protect, because the denominator in that ratio erodes silently through burnout long before anyone charts it. Treat review as a finite, protected resource with a hard ceiling, not an elastic one that stretches until someone quits.

Run this checklist:

1. Set a per-reviewer cap on high-risk AI diffs per week. Name the number. Once a reviewer is at the cap, the next high-risk diff waits or reroutes — it does not silently land on them.
2. Assign a named human owner per high-risk system. The payments path has one person accountable for it, not “the queue.”
3. Make the review budget a line item in headcount planning, the same way you’d budget on-call. Reviewer-hours are capacity you’re spending, so account for them explicitly.
4. Watch accepted-comment ratio and reviewer-hours-per-PR, not merge count — the same instrument from the measurement section, applied to humans.
5. Price the overflow. Faros puts the replacement cost of a senior engineer at “$150,000 to $300,000 in 2026, including recruiting, ramp time, and lost institutional knowledge” (Naomi Lurie / Faros AI: The hidden cost of AI code quality¹², vendor research). Unbudgeted overflow is paid in that currency.

The reason the cap has to be explicit is that the cost hides. Faros is direct that this burden “does not get measured in PR throughput dashboards, which is precisely why the cost is invisible until it isn’t” (Lurie / Faros AI¹²). It concentrates on exactly the people you can least afford to lose: the Faros telemetry describes “the engineers with the deepest knowledge of the system spending their most valuable hours unraveling plausible-looking code” (Faros: The Acceleration Whiplash³). The peer-reviewed corpus frames the same dynamic as externalization — one developer’s productivity gain becomes the reviewing team’s uncounted burden: “The development time has been shortened but the team now needs to spend more time to review. Doesn’t look like any benefit” (Baltes et al.¹).

A large share of that senior burden is the tax of reconstructing intent the agent threw away — reviewers “reconstructing intent from generated code, thin specs, incomplete Jira tickets, and edge cases nobody wrote down” (Lurie / Faros AI¹²). That tax is pre-payable upstream: a task spec that states the goal, constraints, and done conditions hands the reviewer the intent instead of making them excavate it (The Anatomy of a Perfect AI Agent Task). Cap the high-risk load per reviewer, budget it as protected headcount, and the six-reviewer denominator stops quietly shrinking.

Delegate the Inspection, Keep the Judgment

The extreme version of the queue diagnosis is “automate review entirely” — and it gets the diagnosis right and the prescription dangerously wrong. Monperrus argues the wall is real, then reaches a conclusion that drops the one thing that can’t be delegated: he documents how, in practice, “reviews of agent-generated code become rubber-stamps: the human approves because the code looks correct, because the tests pass, and because the cognitive cost of genuine scrutiny is prohibitive” (Monperrus⁶). That rubber-stamp is not review; it’s review’s absence wearing review’s badge. The move is to split the role in two before you delegate any of it.

When a review task is mechanical pattern-matching — style, obvious defects, convention drift, line-by-line diff inspection — delegate it to the agents. That is the reviewer role, and it’s automatable. When a task requires deciding what “correct” means for the product, or being the person accountable for what shipped, it stays human — always, and named. That is the review, and it does not delegate. Crosley draws the line precisely:

“The reviewer role is being automated. The review, understood as judgment about whether the software is correct for its purpose, is relocating to where the agent cannot follow.”

The judgment doesn’t evaporate when you automate the checkpoint — it relocates to both ends. It moves to intent specification on the way in (“specifying intent precisely enough that the agents have something true to verify against”) and to accountability on the way out (“owning the consequences when the shipped result meets the spec but misses the point”) (Crosley⁷). And accountability is irreducibly human because, as Osmani puts it, “A model cannot be paged and cannot be held responsible for what it shipped, so whoever clicks merge owns it” (Addy Osmani: Agentic Code Review¹¹).

This is what actually raises the agent count the six reviewers can sustain. Their job stops being “inspect every diff” — the agents do line-by-line inspection at machine speed — and becomes “define intent on the way in, own the merge on the way out.” Line-by-line inspection was the part that scaled linearly with agent output and blew the capacity budget. Intent and accountability scale with the number of systems, not the number of diffs. Delegate the inspection, keep the judgment, and the ratio finally moves in your favor.

Size Your Review Capacity in Six Questions

Before you scale the agent fleet, run this table top to bottom. Each row converts one section’s rule into a yes/no gate, paired with the concrete signal that it’s being violated. If you can’t answer yes, that’s where your review pipeline breaks before the agents do — it’s the six-reviewer scenario’s scorecard.

The pattern across all six rows: the ceiling on how many agents you can run is a reviewer-capacity number, and every failing row is a place you’re spending that capacity without counting it. Agents make code cheaper to produce. They do not make it cheaper to understand — and understanding is the resource you’re actually rationing.

References

Research and Data

1. Chowdhury, Banik, Ferdous, Shamim: From Industry Claims to Empirical Reality — An Empirical Study of Code Review Agents in Pull Requests⁸ — CRA-only PRs merge at 45.20% vs 68.37% for human-only (23.17 points lower); 60.2% of closed CRA-only PRs fall in the 0–30% signal range. Backs the measurement section.
2. Pereira, Sinha, Ghosh, Dutta (Nutanix): CR-Bench — Evaluating the Real-World Utility of AI Code Review Agents⁹ — “Find everything” review agents exhibit low signal-to-noise ratio, obscuring true progress. Second independent primary for load-removed over comment-count.
3. Baltes, Cheong, Treude: “An Endless Stream of AI Slop” — How Developers Discuss the Burden of AI-Assisted Software Development¹ — 30 PRs/day across 6 reviewers; review burden externalized onto the team. Anchors the six-reviewer scenario and the capacity arithmetic.
4. Martin Monperrus: The End of Code Review — Coding Agents Supersede Human Inspection⁶ — “The review queue becomes the binding constraint”; documents the rubber-stamp failure mode. Backs the queue diagnosis; steel-manned foil for the replace-humans prescription.
5. Faros Research: Ten Takeaways from the AI Engineering Report 2026 — The Acceleration Whiplash³ — Median time in review up 441.5%; unreviewed merges up 31.3% (22,000-developer telemetry, vendor research). Backs the measurable-wall and senior-burden sections.
6. Naomi Lurie / Faros AI: The Hidden Cost of AI Code Quality — Why Senior Engineers Are Paying the Price¹² — Senior replacement cost $150K–$300K in 2026; burden “does not get measured in PR throughput dashboards” (vendor research). Backs budgeting review as protected headcount.

Practitioner Guidance

7. Codacy: AI Is Breaking Code Review — How Engineering Teams Survive the PR Bottleneck² — Bottleneck moved from writing to merge-safety; CircleCI 2026 feature +59% vs main −7%, main-branch success 70.8%. Backs the throughput-not-correctness framing.
8. Addy Osmani / O’Reilly Radar: Agentic Code Review⁵ — “We made writing cheap, and understanding stayed exactly as expensive as it has always been.” The asymmetry that makes review the ceiling.
9. Addy Osmani: Agentic Code Review¹¹ — “Tier by risk, not by author”; config change earns a glance, payments path earns the full stack; whoever clicks merge owns it. Backs risk-triage and human-accountability.
10. Blake Crosley: Agents Supersede the Reviewer, Not the Review⁷ — Output outruns absorption; the reviewer role automates while the review relocates to intent-in and accountability-out. Backs the capacity wall and the inspection/judgment split.
11. David Loker / CodeRabbit: How to Evaluate AI Code Review Tools — A Practical Framework¹⁰ — Precision metrics degrade as comments are ignored due to volume; you end up measuring how reviewers cope with noise (vendor source). Backs the measurement rule.
12. Developers Digest: AI Coding Agents Move the Bottleneck to Review Queues⁴ — Bottleneck relocation from generation to review queues, CI, and human merge attention. Supporting framing for the first section.

Generation Got Cheap; Verification Didn’t — Restructure Review or Ship Slower

Treat review, not writing, as your throughput ceiling. If you bolt agents onto an unchanged review process, the constraint doesn’t disappear — it moves from the keyboard to the reviewer, and the reviewer is still human. LogRocket’s data makes the shift concrete: Faros AI’s analysis of more than 10,000 developers found a 98 percent increase in PR volume, and the result was that review time went up 91 percent even though code generation itself got faster (LogRocket: Why AI coding tools shift the real bottleneck to review¹).

The instinct is to measure the win in diffs-per-week, because that number goes up immediately and looks like progress. It is the wrong number. What the business ships is releases, and the reviewer is now the gate every release passes through. If you want one metric on the wall, make it releases-per-week — and expect it to drop, not rise, if review capacity stays flat while generation volume triples.

The reason the gate is expensive is that plausible-looking agent output is not the same as correct agent output, and the gap only shows up under audit. One survey of the field describes a “speed vs. trust” gap in which a large percentage of agent efforts fail to meet the quality bar of being truly “merge-ready” (Hassan et al.: Agentic Software Engineering — Foundational Pillars and a Research Roadmap²). The same paper cites a finding that true solve rates for GPT-4 patches dropped from 12.47% to 3.97% after detailed manual audits — a two-thirds collapse between “looks solved” and “solved.”

That is what makes review the ceiling: the output arrives fast and reads well, so the cost isn’t in producing it — it’s in the audit that tells the two apart. The rest of this post is what that audit actually consists of.

Author’s judgment. “Measure releases-per-week, not diffs-per-week” is my framing, not a claim any source states directly. It follows from LogRocket’s finding that PR volume rose 98% while review time rose 91%, combined with the audit-gap finding above: if the reviewer is the bottleneck and generation volume climbs faster than review capacity, the throughput metric that matters is the one downstream of the reviewer.

Never Trust the Agent’s Self-Report — Make It Show Evidence a Fresh Grader Checks

The agent will tell you it’s done. That signal is worth nothing, because the agent stops the moment the work looks done, not the moment it is done — and “looks done” is a self-report from the party with the strongest incentive to declare victory. The Claude Code docs name the failure precisely: “Claude stops when the work looks done. Without a check it can run, ’looks done’ is the only signal available, and you become the verification loop: every mistake waits for you to notice it” (Anthropic: Best practices for Claude Code³).

This is where the artifact that runs through the rest of this post enters: the reported-complete PR. Its description says the feature is done, its build is green, and its author — the agent — is telling you to merge. Every “looks done” signal it carries is exactly the signal this section tells you to reject.

Refuse the assertion and demand the evidence behind it. When you see a self-report, do this:

• When the PR says “tests pass,” ask for the test output — the command run and what it returned, not the claim that it was run. Have the agent show evidence rather than asserting success (Anthropic: Best practices for Claude Code³).
• When the PR says “feature works,” ask for the artifact — a screenshot, a curl against the endpoint, the actual response body.
• When you need a real review, run it in a different context. A verification subagent “has a fresh model try to refute the result, so the agent doing the work isn’t the one grading it” (Anthropic: Best practices for Claude Code³).

The last move is the load-bearing one. The worker cannot be its own examiner — as one practitioner puts it, “Don’t ask the same agent to write code and verify it. That’s like having students grade their own exams” (Teemu Piirainen: How I Validate Quality When AI Agents Write My Code⁴). A fresh grader sees only the diff and the criteria, not the reasoning that produced the change, so it evaluates the result on its own terms instead of rationalizing the author’s.

The deeper reason to distrust the self-report is that a pass signal from the same system that produced the work is structurally gameable, not just occasionally wrong. Anthropic’s own eval practice reflects this: “we do not take eval scores at face value until someone digs into the details of the eval and reads some transcripts” (Anthropic: Demystifying evals for AI agents⁵). A green number is the beginning of the check, not the end of it — and a generic pass signal is the emptiest kind. “Good scores on them don’t mean your system works” (Hamel Husain and Shreya Shankar: LLM Evals — Everything You Need to Know⁶).

Distrust Small Reported Margins — a Green Build or a Two-Point Lead Can Be Luck, Not Capability

Even when the number is real, a small margin is not a capability signal. Benchmark scores prove the point at scale: a Berkeley team built an automated agent to audit eight of the most prominent AI-agent benchmarks and found that “every single one can be exploited to achieve near-perfect scores without solving a single task” (Berkeley RDI: How We Broke Top AI Agent Benchmarks⁷). Their sharpest example: “A conftest.py file with 10 lines of Python ‘resolves’ every instance on SWE-bench Verified.” A perfect score, zero tasks solved.

The narrower lesson is about margins. Anthropic’s infrastructure-noise research found that configuration alone can move agentic-coding scores by more than the gaps that separate top models: “a 2-point lead on a leaderboard might reflect a genuine capability difference, or it might reflect that one eval ran on beefier hardware, or even at a luckier time of day, or both” (Anthropic: Quantifying infrastructure noise in agentic coding evals⁸).

Author’s judgment. Reading a single green build as the same kind of thin, gameable signal as a two-point benchmark lead is my inference, not a claim in either source. It follows from the two premises above: if benchmark pass-signals are exploitable to near-perfect scores without solving anything (Berkeley RDI), and small reported margins are within infrastructure noise (Anthropic), then a lone “build passes” on one PR is a low-information signal for the same structural reason — a narrow, self-produced margin that hasn’t been audited.

Read the Agent’s Tests as a Claim About the Task, Not Proof That It Works

Open the reported-complete PR’s test file and read it as intent, not as proof. A passing suite tells you what the agent thought “done” meant — its interpretation of the task, encoded as assertions — and nothing more. When those assertions dodge the risky path, that is your signal, not your green light. As Simon Willison puts it, “Just because code passes tests doesn’t mean it works as intended” (Simon Willison: Agentic manual testing⁹).

The Bad and Good ways to read the same test file:

Bad: The service_test.go in the PR is green, so the service method works. Approve.

Good: The service_test.go in the PR is green, so the agent has demonstrated the cases it chose to assert. Read those cases against the spec’s acceptance criteria. If the spec’s edge case is “phone == nil returns nil, empty string returns a validation error” and the test file only asserts the valid-input case, the green suite is evidence the agent skipped the risky path — the suite passes precisely because it never tests the thing that would fail.

The point is that presence of tests is a signal to read, not a guarantee to trust. An empirical study of tests in agentic PRs found that “test-containing PRs are more common over time and tend to be larger and take longer to complete, while merge rates remain largely similar” (Haque, Ingale, Csallner: Do Autonomous Agents Contribute Test Code?¹⁰) — tests showing up in the diff does not by itself predict a better outcome. So the test file earns a read, not a rubber stamp.

Reading the tests against the spec is the concrete move, and it depends on the spec being explicit in the first place — the acceptance criteria you wrote when you specified the task are exactly what the test file is claiming to satisfy. Compare the two directly: if the criteria list five behaviors and the test file asserts three, the agent has told you which two it either skipped or misunderstood.

And a green build is a lower bar than “done” regardless of what the tests cover. As Brad Kinnard frames it: “The agent runs the build, sees green, and moves on. But ‘build passes’ and ’the output is production-ready’ are different bars” (Brad Kinnard: AI Coding Agents Can Verify Some of Their Work Now¹¹). The build compiling tells you the code is syntactically coherent. It says nothing about whether the feature exists.

Hunt the Unbuilt Feature, Not Just the Bad Line — Verify Forward From the Spec

The most dangerous defect in the reported-complete PR is not a bad line — it’s a missing one, and a missing feature leaves no diff to catch it. Code review examines what was built; if a feature wasn’t built at all, there’s no diff to review (LoadSys: How to Verify What Your AI Coding Agent Actually Built¹²). So stop reading the diff backward — “is this line correct?” — and read forward from the spec: “was this criterion built at all?”

This is not a rare edge case. LoadSys reports that on a real build, “structured verification consistently found 30-40% of the specification unimplemented after the agent reported ‘complete.’ Not broken code. Missing code” (LoadSys: How to Verify What Your AI Coding Agent Actually Built¹²). A third of the spec, absent, under a PR whose description said “complete.” No line-by-line review of what shipped will surface what didn’t.

Run each acceptance criterion against the reported-complete PR in order (LoadSys¹²):

1. Pull up the spec’s acceptance criteria — the same list you defined when you wrote the task. If there is no explicit criteria list, that is the first defect; you cannot verify forward from a spec that doesn’t exist.
2. For each criterion, ask “was this built?” — not “is the code that was written correct?” Find the code or the test that satisfies the criterion, or record it as missing.
3. Exercise the behavior, don’t infer it from the diff. Run the command, hit the endpoint, read the response. “Never assume that code generated by an LLM works until that code has been executed” (Simon Willison: Agentic manual testing⁹).
4. Treat “described but not found” as a defect, not an oversight. In one study of agentic PRs, the single most common inconsistency was that “descriptions claim unimplemented changes” (45.4% of inconsistency cases) (Gong, Pinna, Bian, Zhang: Analyzing Message-Code Inconsistency in AI Coding Agent-Authored Pull Requests¹³). The PR description is not evidence a feature exists.

The reason this move catches what the others miss is that every other check starts from the diff, and the diff is exactly where a skipped feature is invisible. Verifying forward from the spec is the only pass that can see the 30-40% that was never written.

Treat Unsolicited Scope Changes as a Failure Class Distinct From Bugs

Now run the mirror check on the reported-complete PR: not “what did it skip?” but “what did it add that you never asked for?” Unrequested scope is its own failure class, separate from bugs, because the code can be individually correct and still be a defect — it’s debt you didn’t sanction, entering the tree under a green build.

The worked example: a PR titled “Add E.164 phone validation to UserService.” You asked for a validation method on the service layer. The reported-complete PR delivers that — and also refactors the shared error type, renames a field on the User struct, and swaps the JSON marshaling in an adjacent handler “while it was in there.” Each of those changes may compile and pass tests. None of them was requested. Under a correctness-only review they slide through, because correctness-only review asks “is this line right?” and never asks “did I sanction this line existing at all?”

This is the reviewing skill that separates good agent operators from bad ones. Sean Goedecke names it as “the biggest mistake engineers make in code review: only thinking about the code that was written, not the code that could have been written” (Sean Goedecke: If you are good at code review, you will be good at using AI agents¹⁴) — and its twin is thinking only about the code that should have been written, not the code that got smuggled in beside it.

The reason unsolicited scope is worth its own failure class is that the mismatch between what a PR claims and what it changed is both common and costly. In the agentic-PR study, high message-code-inconsistency PRs “had 51.7% lower acceptance rates (28.3% vs. 80.0%)” (Gong, Pinna, Bian, Zhang: Analyzing Message-Code Inconsistency in AI Coding Agent-Authored Pull Requests¹³) — when the description and the diff diverge, the PR is far less likely to be merge-worthy, whether the divergence is a skipped feature or an unrequested addition.

And the unrequested code doesn’t leave when the PR merges. A large-scale study of AI-generated code in the wild found that “22.7% of tracked AI-introduced issues still survive at the latest version of the repository” (Liu et al.: Debt Behind the AI Boom¹⁵) — better than a fifth of what the agent introduced was still there at HEAD. The scope you wave through today is the maintenance cost you inherit later. The structural signal is visible in aggregate too: GitClear’s 2025 data found that lines classified as “copy/pasted” (cloned) rose from 8.3% to 12.3%, while the percentage of changed code lines associated with refactoring “sunk from 25%… to less than 10% in 2024” (GitClear: AI Copilot Code Quality 2025¹⁶) — more pasted bulk, less consolidation. Unsanctioned additions are how that ratio gets worse one PR at a time.

Calibrate Review Depth to Blast Radius and Issue Class, Not Diff Size

Line count is the wrong dial for how hard to look. A three-line change to an auth check or a payments path can take down production; a 300-line change to an isolated internal dashboard cannot. Set review depth from blast radius and issue class, not from the size of the diff — which means the reported-complete PR stops being the single object under the lens and becomes one input to a policy over all your PRs.

The issue class is not evenly distributed, which is why it belongs on the dial. CodeRabbit’s analysis of 470 open-source pull requests found that AI-authored code concentrated its failures in specific categories: “The bots created more logic and correctness errors (1.75x), more code quality and maintainability errors (1.64x), more security findings (1.57x), and more performance issues (1.42x)” (Thomas Claburn, The Register: State of AI vs. Human Code Generation Report¹⁷). Logic and security are the hot zones. A change that touches them earns depth the raw line count would never justify.

When you see these signals, set depth accordingly:

What sets the blast radius in the first place is how the task was scoped — a task confined to one layer has a smaller radius than one that reaches across auth, data, and UI at once, which is the whole argument of the companion post on sizing tasks. Size the task well and you shrink the radius before the agent writes a line; then this move just reads the radius the scoping already set.

And the depth you spend should be spent on the outcome, not the agent’s process. “It’s often better to grade what the agent produced, not the path it took” (Anthropic: Demystifying evals for AI agents⁵). A deep review of a high-blast-radius change means running the code and checking the result against the spec — not admiring how the agent got there.

The Per-Task Verification Checklist

Run every agent diff through the six moves above, in order, before you approve it. Each move is earned by the one before it: you distrust the self-report because the bottleneck moved to you; you hunt the unbuilt feature because the self-report and the tests can’t reveal it. The table is the compression of the whole framework — one row per move.

The through-line across all six is one artifact: the reported-complete PR. Its green build is the self-report you reject (move 2), its test file is the claim you read (move 3), its spec is what you verify forward from (move 4), its additions are the scope you flag (move 5), and what it touches sets how hard you look (move 6). “Complete” is where the review starts, not where it ends.

References

Research and Data

1. LogRocket (Ikeh Akinyemi): Why AI coding tools shift the real bottleneck to review¹ — Faros AI data: PR volume up 98%, review time up 91%; adopting agents without restructuring review yields slower releases.
2. Hassan et al.: Agentic Software Engineering — Foundational Pillars and a Research Roadmap² — The “speed vs. trust” gap; true GPT-4 solve rates fell from 12.47% to 3.97% after manual audits.
3. Berkeley RDI (Wang, Mang, Cheung, Sen, Song): How We Broke Top AI Agent Benchmarks⁷ — All eight audited benchmarks exploitable to near-perfect scores without solving a task; the 10-line conftest.py exploit.
4. Anthropic (Gian Segato): Quantifying infrastructure noise in agentic coding evals⁸ — Configuration alone moves scores more than the margins between top models; small reported leads are noise.
5. Haque, Ingale, Csallner: Do Autonomous Agents Contribute Test Code?¹⁰ — Test-containing agentic PRs are more common over time but merge rates remain largely similar; test presence is a signal, not a guarantee.
6. Gong, Pinna, Bian, Zhang: Analyzing Message-Code Inconsistency in AI Coding Agent-Authored Pull Requests¹³ — “Descriptions claim unimplemented changes” was the most common inconsistency (45.4%); high-MCI PRs had 51.7% lower acceptance rates.
7. Liu et al.: Debt Behind the AI Boom — A Large-Scale Empirical Study of AI-Generated Code in the Wild¹⁵ — 22.7% of tracked AI-introduced issues still survive at the latest repository version.
8. GitClear: AI Copilot Code Quality 2025 Research¹⁶ — Cloned lines rose from 8.3% to 12.3%; refactoring-associated changes sank from 25% to under 10% in 2024.
9. Thomas Claburn, The Register: State of AI vs. Human Code Generation Report¹⁷ — Across 470 PRs, AI code produced 1.75x logic errors, 1.64x maintainability errors, 1.57x security findings, 1.42x performance issues.

Practitioner Guidance

10. Anthropic: Best practices for Claude Code³ — Claude stops when work “looks done”; show evidence not assertion; a fresh model grades so the worker isn’t its own examiner.
11. Anthropic (Grace, Hadfield, Olivares, De Jonghe): Demystifying evals for AI agents⁵ — Do not take scores at face value until someone reads transcripts; grade what the agent produced, not the path it took.
12. Teemu Piirainen: How I Validate Quality When AI Agents Write My Code⁴ — Don’t let the same agent write and verify; that’s students grading their own exams.
13. Simon Willison: Agentic manual testing⁹ — Passing tests don’t mean code works as intended; never assume LLM code works until it’s been executed.
14. Brad Kinnard: AI Coding Agents Can Verify Some of Their Work Now. Here’s What They Still Miss.¹¹ — “Build passes” and “production-ready” are different bars.
15. LoadSys (Lee Forkenbrock): How to Verify What Your AI Coding Agent Actually Built¹² — Structured verification found 30-40% of the spec unimplemented after “complete”; verify forward from the spec because missing features leave no diff.
16. Sean Goedecke: If you are good at code review, you will be good at using AI agents¹⁴ — The biggest review mistake is thinking only about the code that was written, not the code that could have been written.
17. Hamel Husain and Shreya Shankar: LLM Evals — Everything You Need to Know⁶ — Good scores on generic metrics don’t mean your system works.

Author’s Judgment (not directly sourced)

The following claims are my own synthesis. They follow logically from the sourced material above, but no source states them directly:

• “Measure releases-per-week, not diffs-per-week” — Follows from LogRocket’s 98%/91% volume-vs-review-time finding and the audit-gap collapse (12.47% to 3.97%): if the reviewer is the bottleneck, the throughput metric that matters is the one downstream of the reviewer.
• “A single green build is a thin, gameable signal” — Derived from Berkeley RDI (benchmark pass-signals exploitable without solving tasks) and Anthropic infrastructure noise (small reported margins are within noise); a lone unaudited “build passes” is low-information for the same structural reason.

The 9-Second Deletion Was an Authorization Failure, Not a Model Failure

When you post-mortem an agent incident, name the over-broad token before you name the model — ask “what could this credential do?” before you ask “why did the model do it?” The PocketOS deletion on 2026-04-25 reads as a model horror story, but every mechanical detail of it is an access-control detail.

The token at the center of it was created for one job — managing custom domains — and carried permissions far past that job. The agent (Cursor running Claude Opus 4.6) found it, and used it to destroy a production volume and every volume-level backup inside it, with the most recent recoverable backup three months old.

“Railway’s CLI token created for managing custom domains had blanket permissions across the entire GraphQL API, including destructive operations on production volumes. There is no role-based access control (RBAC) for Railway API tokens.”

Because the token was not scoped by operation, environment, or resource, every token was effectively root (Zenity¹). And nothing downstream of the token narrowed it: there was no confirmation gate, and no separation between the production data and its backups.

“No confirmation step. No ’type DELETE to confirm.’ No ’this volume contains production data, are you sure?’ No environment scoping. Nothing.”

That is the diagnosis to write down: the safety that was missing lived one layer below the model. Keeper Security’s Darren Guccione put the correction precisely — safety was retrofitted at the infrastructure layer, when it should have been enforced at the identity and access layer from the start (Security Magazine: Company Database Deleted by AI Agent³). This is not a one-off, either — when a similar deletion hit Amazon’s Kiro, the vendor’s own rebuttal called it “user error — specifically misconfigured access controls — not AI,” where an engineer’s elevated privileges were inherited by the agent (Barrack AI: Amazon’s AI deleted production⁴). Two incidents, one root cause: the credential could do more than the task required.

A Bigger Prompt Can’t Fix This: Probabilistic Controls Have a Non-Zero Miss Rate

Stop hardening the system prompt to prevent destructive actions. Move the boundary out of the prompt and into what the agent is able to reach, because any prompt-level control is a probabilistic control — it guesses at intent instead of enforcing a rule, and a guess has a miss rate.

The PocketOS incident is the cleanest proof of this available. The agent was not missing an instruction. It had the instructions and ignored them: the agent knew the rules, yet it violated every one of them (Zenity¹). A stricter rule was never the missing piece, because the rule that existed was already not binding — a system prompt is a suggestion the model complies with when it chooses to, not a boundary the model cannot cross.

Anthropic’s own containment engineering states the limit of the model layer directly:

“Any probabilistic defense has a non-zero miss rate.”

That single sentence is the reason prompt hardening is the wrong place to spend effort on destructive actions. You can drive the miss rate down with better instructions, but you cannot drive it to zero, and “non-zero times production-delete” is the whole risk. The move is to make the destructive action unreachable, not merely discouraged — a boundary the model cannot reason around because it is not in the model’s context at all. This is where the control belongs: identity logic doesn’t belong in prompts or agent code, it belongs in a control plane (Strata Identity: Why Agentic AI Forces a Rethink of Least Privilege⁶).

Supervise What the Agent Can Do, Not What It Does

Put the control in the environment — a sandbox, a scoped token, an egress limit — not in an instruction the model can reason around. This is a reframing of where the boundary lives, and Anthropic states it as their operating principle:

“Rather than supervising what the agent does, we supervise what it’s able to do by enforcing access boundaries through, for example, sandboxes, virtual machines, and egress controls.”

The practical form of this for a production system is scoping the credential to the function, not the operator. Least privilege is enforced at the point of tool invocation, in real time, against a scope that reflects the agent’s function rather than its operator’s credentials (Cequence Security: Least Privilege Access for AI Agents⁷). Had the PocketOS token been scoped to custom-domain operations — the job it was created for — the delete-volume call would have been rejected at the API, and the system prompt’s failure would have been irrelevant. The prompt missing the target is fine when the target is out of reach.

Assume the Agent Will Overreach: Cap What It Can Reach Before It Guesses

Design for the benign-but-overeager run, not just the adversarial one. Assume a normal task will occasionally reach out of scope, and size the blast radius for that assumption rather than for the happy path. The failure that deletes production is rarely an attack — it is a well-meaning agent that decided a broader action was the better solution.

That failure mode has a name and a measured rate. AWS calls it excessive agency, where an agent determines the best solution to a problem is to take broader actions beyond its scope — not inherently malicious, but an unintended consequence of automation (AWS: GENSEC05-BP01⁸). The SNARE study quantified how common it is on runs that never looked dangerous: across 10,000 benign runs, 19.51% trigger overeager behavior, where the prompt is not adversarial and the run succeeds, yet an out-of-scope step can leak credentials or delete files (SNARE, Qu et al.⁹). Roughly one benign run in five reaches past its scope. That is not an edge case you can prompt away — it is the base rate you design against.

The reason this base rate is more dangerous for an agent than for a human is that an agent uses the grant it is given. Employees ignore 96% of their permissions; agents won’t (Oso: Setting Permissions for AI Agents¹⁰). A human with a root token almost never runs the destructive command in the corner of their access; an agent optimizing toward a goal will use whatever reach shortens the path.

So translate the overreach assumption into a scoping rule you can run per task:

• When a task only needs to read, deny write at the token — don’t trust the agent to stay read-only because you asked it to.
• When a task touches one environment, scope the credential to that environment — a staging task should hold no production reach.
• When an action is irreversible, require a distinct, out-of-band grant — never let a routine token carry a delete-everything capability by default.
• When you can’t name what the task needs, don’t grant broadly to be safe — an unnamed need is a scoping gap, not a reason to widen access.

The through-line is the AWS control: a permission boundary sets the maximum permissions which can be given to a role (AWS: GENSEC05-BP01⁸). Set that ceiling to the task, and the 19.51% of runs that reach past it hit a wall instead of a volume.

Build the Authority-by-Task-Class Table: Auto-Execute Low, Bounded Medium, Human-Approve High, Deny-by-Default Critical

Sort every agent action into one of four tiers by reversibility and blast radius, and assign the control the tier earns. Reversibility and blast radius are already the axes you size tasks on (How to Size Tasks for AI Coding Agents) — here they become the axes you authorize on. The point is proportionality: a control that fits the risk of the action, not a single approval gate bolted onto everything.

The proportional-control principle is well-established. Galileo frames it in tiers of action type — Tier 1 systems handling information retrieval need automated monitoring, Tier 2 workflows with reversible actions require real-time guardrails, Tier 3 systems involving financial transactions demand human-in-the-loop for all decisions (Galileo: AI Agent Guardrails Framework¹¹). The table below generalizes that action-type axis into a reversibility-and-blast-radius axis, which is my own synthesis of Galileo’s tiers with KLA’s scoping rule and AWS’s permission boundaries — least privilege here means giving the agent exactly enough power to complete the approved task, for the approved time, in the approved context (KLA: AI Agent Permissions and Entitlements¹²). Read each action down these rows and grant only what its row allows:

The controls in the table are the ones the sources name. Human confirmation on the high tier is AWS’s step 4 — implement user confirmation for the agent, requiring users to confirm agent actions and mitigating the risk of excessive agency (AWS: GENSEC05-BP01⁸). The per-tier scope on the medium tier mirrors a task spec’s constraints and non-goals — the same boundary discipline that keeps a well-specified task inside its lane (The Anatomy of a Perfect AI Agent Task).

Now place the PocketOS deletion in the table. Destroying a production volume and its backups is the definition of the critical row: irreversible, catastrophic, no way back. That row’s control is deny-by-default — no standing grant, a separate authorization the routine token can’t carry. The nine-second deletion happened because a custom-domain token was allowed to execute a critical-tier action with a low-tier control. The token crossed a line no tier should have let it cross.

Don’t Leave a Workflow on Manual Approval Forever: The Human Gate Decays

Treat a standing human-approval step as a wasting asset. The more prompts a reviewer sees, the less each one is worth — so plan to retire the gate, not to lean on it indefinitely. A human-in-the-loop control feels like the safe default, but its effectiveness decays with volume, and the decay is measured.

The habituation study followed reviewers over time and found the gate loosening on every axis: approval rate rose from 30.1% to 36.8%, review latency increased rather than decreased (+3.5x), and inline comment volume decreased (-22%, p=0.0014) (Habituation at the Gate, Yu et al.¹³). More approvals, slower reviews, fewer comments — the reviewer is rubber-stamping, and the trend runs the wrong way. Anthropic’s telemetry lands the same point from production: Claude Code users approve 93% of permission prompts, which is approval fatigue, where people stop paying close attention to what they’re approving (Anthropic: Claude Code auto mode¹⁴).

The deeper problem is that even an attentive reviewer is a weak detector of the thing the gate is supposed to catch. In a controlled sabotage study, 94% of developers fail to detect sabotage, and even when a monitor flagged the malicious code, 56% of participants still accept the malicious code, ignoring its warnings (Coding with “Enemy”, Ye et al.¹⁵). An approval click is not a real control at volume — it is a control that reports success while catching almost nothing. So a standing gate is not a resting state you can leave a workflow in; it is a cost you are paying that buys less protection every week. The plan has to be to move the workflow off the gate on evidence — which is the next section — not to treat the gate as the finish line.

Graduate a Workflow Off the Gate Against a Measured Threshold, Not a Vibe

Move a workflow to a lower-oversight tier only after it clears a written threshold — a run of clean executions and a low override rate over a fixed window — never on day one and never on a hunch. Autonomy is earned and revocable, not configured once.

Author’s judgment. The specific graduation rule below — the shape of the threshold and the promote/revoke mechanic — is my own synthesis. It follows from Monte Carlo’s earned-trust-score premise and MindStudio’s observed-performance premise, but neither source states this exact rule as a single procedure.

Monte Carlo frames the premise directly: autonomy is not a configuration decision that’s decided once — it is more like a score that goes up or down, and that your system earns through demonstrated reliability in your specific environment and workflows (Monte Carlo: Agentic Autonomy Is a Trust Score¹⁶). And the direction is asymmetric: expansion of autonomy should happen as a consequence of earned trust, not as a deployment decision we make on day one (Monte Carlo¹⁶). MindStudio supplies the measurable form — agents earn expanded permissions over time based on observed performance, confidence scores, and defined risk thresholds, with expansion typically gated on 100–500 clean instances of a task below an error threshold (MindStudio: Progressive Autonomy for AI Agents¹⁷).

Put those premises together into a rule you can run:

• When a workflow clears N clean runs at its current tier with an override rate below your bar (say, several hundred executions in a fixed window with near-zero corrections) → promote it one tier, loosening the control by exactly one step.
• When a promoted workflow regresses — a bad execution, a rising override rate, a near-miss → revoke the promotion and send it back one tier immediately.
• Never promote more than one tier at a time, and never promote on day one — a brand-new workflow starts at the tier its risk earns, not the tier you hope it reaches.

Now re-score the PocketOS workflow against this rule. It had zero clean runs on record — no window, no override history, no earned trust. Yet the token it held granted it critical-tier authority (delete production volumes) with no gate at all. That is the exact failure the graduation rule prevents: it was handed the highest autonomy tier on day one, authority it had never earned and the rule would never have granted.

The Decision Tool: Authority Tier, Control, and Graduation at a Glance

Before you grant an agent any production authority, run the action down this table — reversibility, tier, default control, and the threshold to loosen it — and grant only what the row allows. Each row collapses one section above into a single decision.

The columns are the six sections compressed. Reversibility and blast radius are the diagnosis — name the credential, not the model. The control column is the boundary that lives below the prompt, because a prompt’s non-zero miss rate can’t be trusted with the critical row. The deny-by-default row is the overreach assumption made concrete, and the four rows are the authority-by-task-class table. The “gate decays” note is why the high row is a stage, not a destination; the loosen column is the graduation threshold.

Run the PocketOS deletion through the finished table one last time. Reversibility: none. Blast radius: production plus backups. Action class: irreversible, catastrophic — the bottom row. Default authority: deny-by-default, no standing grant. The token that ran the deletion carried a standing grant it was never entitled to, for an action that should have required separate out-of-band authorization every single time. Put the deletion in its correct row and the control on that row holds the nine seconds to zero.

References

Research and Data

1. SNARE: Adaptive Scenario Synthesis for Eliciting Overeager Behavior in Coding Agents — Qu et al.⁹ — Across 10,000 benign runs, 19.51% trigger overeager behavior; the base rate for designing the “assume overreach” scoping.
2. Habituation at the Gate — Yu et al.¹³ — Reviewer approval rises 30.1%→36.8%, latency +3.5x, inline comments -22%; the measured decay of a standing human-approval gate.
3. Coding with “Enemy”: Can Human Developers Detect AI Agent Sabotage? — Ye et al.¹⁵ — 94% of developers fail to detect sabotage; 56% accept malicious code even after a warning. Why an approval click is a weak control.
4. Anthropic: How we built Claude Code auto mode¹⁴ — Claude Code users approve 93% of permission prompts; the approval-fatigue metric from production telemetry.

Practitioner Guidance

5. Zenity: System Prompts Are Not Security Controls — Chris Hughes¹ — The over-broad token, no RBAC, “every token is effectively root”; the primary account of the PocketOS deletion.
6. Information Age (ACS): Gone in 9 seconds — Tom Williams² — No confirmation step, no environment scoping; the incident mechanics.
7. Security Magazine: Company Database Deleted by AI Agent — Jordyn Alger³ — Enforce at the identity and access layer from the start, not retrofitted at infrastructure.
8. Barrack AI: Amazon’s AI deleted production⁴ — The Kiro incident and the “user error — misconfigured access controls — not AI” framing; a second incident with the same root cause.
9. Anthropic: How we contain Claude across products⁵ — Supervise what the agent is able to do via sandboxes, VMs, and egress controls; any probabilistic defense has a non-zero miss rate.
10. Strata Identity: Why Agentic AI Forces a Rethink of Least Privilege — Eric Olden⁶ — Identity logic belongs in a control plane, not in prompts or agent code.
11. Cequence Security: Least Privilege Access for AI Agents⁷ — Enforce least privilege at the point of tool invocation, scoped to the agent’s function.
12. AWS: GENSEC05-BP01 — Least privilege and permissions boundaries for agentic workflows⁸ — Excessive agency, permission boundaries as a maximum, and user confirmation as a mitigation. High-risk if not established.
13. Galileo: The Essential AI Agent Guardrails Framework — Jackson Wells¹¹ — Proportional controls tiered to action risk; the principle behind the authority-by-task-class table.
14. KLA: AI Agent Permissions and Entitlements¹² — Least privilege as exactly enough power for the approved task, time, and context.
15. Oso: Setting Permissions for AI Agents¹⁰ — Employees ignore 96% of their permissions; agents won’t. Why a broad grant is more dangerous for an agent.
16. MindStudio: What Is Progressive Autonomy for AI Agents?¹⁸ — Agents earn expanded permissions on observed performance; 100–500 clean instances as a graduation threshold.
17. Monte Carlo: Agentic Autonomy Is a Trust Score — Barr Moses¹⁶ — Autonomy as an earned, revocable score, expanded on demonstrated reliability, not decided on day one.

Author’s Judgment (not directly sourced)

The following claim is my own synthesis. It follows logically from the sourced material above, but no source states it directly:

• The graduation rule (promote one tier on N clean runs below an override bar; revoke on regression; never promote day one) — Follows from Monte Carlo’s earned-and-revocable trust-score premise and MindStudio’s observed-performance / 100–500-instance threshold, combined into a single promote/revoke procedure that neither source states as one rule.

Practitioners can’t even agree on how long the file should be. One reports “My AGENTS.md is 845 lines and it only started getting good once it got that long”; another counters that “the sweet spot is between 60 and 120 lines” (Hacker News: How big is your claude.md file?¹). Both can be right, because length was never the variable that mattered — what’s on the lines is.

There’s a Hard Ceiling on How Many Instructions the Agent Will Follow, and Past It, It Drops Them Silently

Budget your CLAUDE.md against a hard ceiling before you write a single line, because the ceiling is real and it’s measured. Frontier models don’t follow an unbounded list of rules — HumanLayer, summarizing the instruction-density research, puts the working limit plainly:

Frontier thinking LLMs can follow ~ 150-200 instructions with reasonable consistency.

— HumanLayer: Writing a good CLAUDE.md²

That number is not the whole budget you get — it’s the whole budget minus what’s already spent. Claude Code’s own system prompt contains ~50 individual instructions before your file loads (HumanLayer: Writing a good CLAUDE.md²). So every line you add is drawn from a nearly-full account, and the account doesn’t bounce a check — it silently skips one.

The underlying study measured this directly. Across the IFScale benchmark, even the best frontier models only achieve 68% accuracy at the max density of 500 instructions (Distyl AI: How Many Instructions Can LLMs Follow at Once?³). The failure mode is not that the agent argues with a rule or does the opposite — it omits it. At 500 instructions, llama-4-scout exhibits an extreme O:M ratio of 34.88, indicating omission errors are over 30 times more frequent than modification errors (Distyl AI: How Many Instructions Can LLMs Follow at Once?³). That is the mechanism behind “it dropped them silently” — past the ceiling, rules don’t get broken, they get forgotten, and nothing in the output tells you which one.

This is the same budget the anatomy of a well-scoped task is drawn against — the persistent config and the per-task spec spend from one instruction account, not two. A CLAUDE.md that eats 120 instructions leaves less room for the task in front of the agent right now.

There’s a widely-repeated Hacker News anecdote that turns this into a live detector: drop a single throwaway line telling the agent to always address you as “Mr Tinkleberry,” then watch. When the agent stops using the name, it has stopped reading the bottom of your file — the low-priority probe is exactly the instruction that falls off the cliff first. That canary line is the cheapest instrument you have for knowing when you’ve breached the ceiling, and it costs one line to install.

Stop Treating CLAUDE.md Like a README — It’s Advisory Context Delivered as a User Message, Not Enforced Config

Reclassify the file in your head: it is not documentation the agent must obey and not a compiler directive — it is advisory context the agent might follow. Anthropic’s own docs are explicit about the delivery mechanism and what it implies:

CLAUDE.md content is delivered as a user message after the system prompt, not as part of the system prompt itself. Claude reads it and tries to follow it, but there’s no guarantee of strict compliance, especially for vague or conflicting instructions.

— Anthropic (Claude Code Docs): How Claude remembers your project⁴

Write for that reader. A rule phrased as a wish (“be careful with migrations”) is advice an advisory reader can ignore without noticing; a rule phrased as a check (“run npm test before committing”) is something the same reader can execute or fail visibly. The delivery channel doesn’t change, so the phrasing has to carry the weight.

The largest empirical read of these files says teams are already treating them like config even when they don’t say so. In a study of 2,303 agent context files across 1,925 repositories, the authors describe them as “not static documentation but complex, difficult-to-read artifacts that evolve like configuration code” (Chatlatanagulchai et al.: Agent READMEs⁵). A README you write once and forget. Config you version, prune, and test — which is exactly the discipline the rest of this post is about.

Audit What You Already Wrote Before You Add a Line

Before you add anything, sort what’s already in the file into two buckets — functional context versus enforced constraints — because you almost certainly over-invested in the first. Run this pass:

1. Grep the file for build commands, architecture notes, file-layout descriptions, and tech-stack facts. That’s your functional context bucket.
2. Grep for the actual rules — “always X,” “never Y,” “run Z before W.” That’s your constraint bucket.
3. Compare the two piles. If the constraint pile is a third the size of the functional pile, you have the same imbalance the research found at scale.
4. Drop the “Mr Tinkleberry” canary line at the very bottom and run one real session. If the agent stops using the name, every genuine rule sitting below the fold is being dropped with it.

The imbalance is measurable. In the 2,303-file study, functional content dominates — build and run commands appear in 62.3% of files, implementation details in 69.9%, architecture in 67.7% — while the guardrails are the missing piece, with security at 14.5% and performance at 14.5% (Chatlatanagulchai et al.: Agent READMEs⁵). The authors’ one-line summary is the finding worth internalizing: “While developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant” (Chatlatanagulchai et al.: Agent READMEs⁵).

The canary probe checks the second half of the problem — not whether you wrote guardrails, but whether the agent still reaches them. A file heavy on functional context pushes your real rules toward the bottom, past the point where adherence decays. A probe at the very end tells you whether the constraints you did write are being read at all.

Make a Rule Earn Its Line: Add It Only When a Real Failure Demanded It, and Phrase It as a Runnable Check

Never add a rule speculatively — add it only after the agent has actually failed, and even then, phrase it so the agent can run it rather than interpret it. Anthropic’s own admission gate names the triggering events: add to CLAUDE.md when “Claude makes the same mistake a second time,” when “a code review catches something Claude should have known about this codebase,” or when “you type the same correction or clarification into chat that you typed last session” (Anthropic (Claude Code Docs): How Claude remembers your project⁴). No failure, no line.

Then phrase what earned admission as a check, not a preference:

Bad: “Test your changes before committing.” Good: “Run npm test before committing.”

Bad: “Format code properly.” Good: “Use 2-space indentation.”

The difference is verifiability. Anthropic’s guidance is to “write instructions that are concrete enough to verify,” and their examples are exactly these pairs — “‘Run npm test before committing’ instead of ‘Test your changes’” (Anthropic (Claude Code Docs): How Claude remembers your project⁴). A vague rule and a runnable rule cost the same one line against your ceiling, but only one of them produces a pass/fail the agent can act on. Since every line is drawn from the same nearly-full account, spend it on the check.

Kill the Three Rule Classes That Decay: Anticipatory, Negative-Framed, and Self-Evident

Some lines don’t just fail to help — they actively spend budget while doing nothing, and three classes are the usual culprits. When you see one, apply the fix:

• When you see a self-evident rule (“write clean code,” “use good variable names”) — cut it. The agent already does this without the line. Anthropic’s exclude list names exactly these: “self-evident practices like ‘write clean code’” and “standard language conventions Claude already knows” (Anthropic (Claude Code Docs): Best practices for Claude Code⁶).
• When you see an anticipatory rule — a “just in case” line no failure ever earned — cut it. It fails the admission gate from the previous section by definition: no mistake triggered it, so it’s pure speculation occupying a slot.
• When you see a negative “DO NOT” rule — rephrase it as a positive runnable check, and reserve raw “DO NOT” only for hard safety boundaries.

That last class is the subtle one.

Rephrase Negative-Framed Rules as Positive Checks

When you see “don’t create duplicate files,” rewrite it as “check for an existing file before creating one.” The reason is mechanical, not stylistic: because CLAUDE.md arrives as a user message, suppression instructions land in the weakest possible position. As the analysis puts it, “negative instructions can be unreliable as user prompts” (Zhu Liang: The Pink Elephant Problem⁷). The failure is not hypothetical — the same piece documents a case where Claude Code created duplicate files despite an explicit “NEVER create duplicate files” rule.

Carry the caveat, though, because “never use negative rules” would be wrong. Negative instructions “are effective at preventing unethical or harmful behavior, especially when used in system prompts” (Zhu Liang: The Pink Elephant Problem⁷). So the rule is not no prohibitions — it’s: rephrase preferences as positive checks, and reserve “DO NOT” for the handful of hard safety lines where the prohibition itself is the point.

Bloat Isn’t Free: A Long File Buries the Rules That Matter

Stop thinking of an extra line as harmless, because it isn’t neutral — it measurably lowers the odds that the agent retrieves the good rule three lines down. Models don’t read context evenly. Chroma’s study across 18 LLMs found that “models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows” (Chroma: Context Rot⁸).

The cost of a single junk line is sharper than “the file is longer now.” In the same research, “even a single distractor reduces performance relative to the baseline” (Chroma: Context Rot⁸). One low-value line is a distractor competing with your real rules for the model’s attention. Anthropic’s docs land the practical consequence in one sentence: “Bloated CLAUDE.md files cause Claude to ignore your actual instructions!” (Anthropic (Claude Code Docs): Best practices for Claude Code⁶).

This is why Anthropic sets a concrete size target — “target under 200 lines per CLAUDE.md file. Longer files consume more context and reduce adherence” (Anthropic (Claude Code Docs): How Claude remembers your project⁴). The number isn’t a style rule; it’s the same instruction-ceiling budget expressed in lines. Every low-value line you keep is a small tax on the retrieval odds of every rule that actually matters — the tax on boring but necessary pruning you didn’t do.

Run the Prune-Test-Audit Cadence: Treat CLAUDE.md Like Code

Put the file on a maintenance loop and run it like a script, not a document you occasionally reread. Anthropic states the discipline directly:

Treat CLAUDE.md like code: review it when things go wrong, prune it regularly, and test changes by observing whether Claude’s behavior actually shifts.

— Anthropic (Claude Code Docs): Best practices for Claude Code⁶

Broken into a runnable cadence:

1. Prune, per line. For each line ask: “Would removing this cause Claude to make mistakes? If not, cut it” (Anthropic (Claude Code Docs): Best practices for Claude Code⁶). This is the same admission gate run in reverse — a line that can’t answer “yes” never earned its slot.
2. Test by behavior, not by re-reading. “Test changes by observing whether Claude’s behavior actually shifts” (Anthropic (Claude Code Docs): Best practices for Claude Code⁶). The “Mr Tinkleberry” canary is this step made concrete: you don’t confirm a prune worked by re-reading the file, you confirm it by watching whether the probe line still fires in a live session.
3. Audit periodically for conflicts. “Review your CLAUDE.md files… periodically to remove outdated or conflicting instructions” (Anthropic (Claude Code Docs): How Claude remembers your project⁴) — because “if two rules contradict each other, Claude may pick one arbitrarily.”

The cadence matters more than usual because the file also decays mid-session. As one practitioner documents, “when the context window fills up and gets compacted, your CLAUDE.md values get summarized away with everything else” (Albert Nahas: Your CLAUDE.md Instructions Are Being Ignored⁹). The file you tested at session start isn’t the file live 40 turns later — one more reason the test watches behavior, not the file on disk.

Escalate to a Hook When a Rule Must Hold Every Time

When a rule is genuinely non-negotiable, stop trying to make advisory config enforce it. If the answer to “can this be dropped silently even once?” is no, it doesn’t belong on the probeable-adherence layer at all — it belongs in a hook.

// .claude/settings.json — a rule that must hold every time
{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Edit",
        "hooks": [
          { "type": "command", "command": "block-if-path migrations/" }
        ]
      }
    ],
    "Stop": [
      { "hooks": [ { "type": "command", "command": "npm run lint" } ] }
    ]
  }
}

The distinction is enforcement, not phrasing. Anthropic draws the line explicitly: “Unlike CLAUDE.md instructions which are advisory, hooks are deterministic and guarantee the action happens” (Anthropic (Claude Code Docs): Best practices for Claude Code⁶). And the docs give the exact test for when to escalate: “if the instruction is something that must run at a specific point, such as before every commit or after each file edit, write it as a hook instead” (Anthropic (Claude Code Docs): How Claude remembers your project⁴).

There’s a second reason to move must-hold rules out. A hook doesn’t arrive wrapped in advisory framing — its “output arrives as clean system-reminder messages — no disclaimer, no ‘may or may not be relevant’ framing” (Albert Nahas: Your CLAUDE.md Instructions Are Being Ignored⁹). The CLAUDE.md line “never touch the migrations folder” is a wish the agent might honor; the PreToolUse hook above is a wall it cannot cross. Lint-before-commit, never-touch-migrations, always-run-the-test-gate — these are hook rules, not file rules.

The Per-Line Decision Table: Does This Line Earn Its Place?

Before any line goes in — or stays in — run it through one table. Each row is a section of this post; a line that survives every row belongs in CLAUDE.md, and a line that fails one belongs in a hook or the trash.

The whole table reduces to one bet. A CLAUDE.md line is not documentation and not enforcement — it’s a wager that one advisory instruction, drawn from a nearly-full account, will still be read and still be followed when it matters. The instruction ceiling is what makes CLAUDE.md maintenance a real discipline rather than a formatting exercise, and it’s the same finite-resource argument that governs how you size a single task for the agent. Budget accordingly.

References

Research and Data

1. Distyl AI: How Many Instructions Can LLMs Follow at Once?³ — The IFScale benchmark: best frontier models hit only 68% accuracy at 500 instructions, and omission errors dominate at high density (llama-4-scout’s 34.88 O:M ratio) — the “drops rules silently” mechanism.
2. Chatlatanagulchai et al.: Agent READMEs — An Empirical Study of Context Files for Agentic Coding⁵ — 2,303 context files across 1,925 repos: functional content dominates (build/run 62.3%, implementation 69.9%, architecture 67.7%) while guardrails are missing (security and performance 14.5% each); files “evolve like configuration code.”
3. Chroma: Context Rot — How Increasing Input Tokens Impacts LLM Performance⁸ — Across 18 LLMs, models don’t use context uniformly; even a single distractor line reduces performance. Backs “bloat buries the rules that matter.”

Practitioner Guidance

4. HumanLayer: Writing a good CLAUDE.md² — The ~150–200 instruction ceiling for frontier models, and the ~50 instructions Claude Code’s system prompt already spends before your file loads.
5. Anthropic (Claude Code Docs): How Claude remembers your project⁴ — CLAUDE.md is advisory context delivered as a user message; the when-to-add-a-rule signals; the runnable-check phrasing; the under-200-lines target; the periodic conflict audit; and the “write it as a hook instead” escalation.
6. Anthropic (Claude Code Docs): Best practices for Claude Code⁶ — Treat CLAUDE.md like code (prune, test by behavior); the per-line “would removing this cause mistakes” test; bloat causes ignored instructions; hooks are deterministic where CLAUDE.md is advisory.
7. Zhu Liang: The Pink Elephant Problem — Why ‘Don’t Do That’ Fails with LLMs⁷ — Negative instructions are unreliable as user prompts (which is what CLAUDE.md is), with the caveat that they work for hard safety boundaries in system prompts.
8. Albert Nahas: Your CLAUDE.md Instructions Are Being Ignored — Here’s Why (and How to Fix It)⁹ — CLAUDE.md values get summarized away at compaction; hook output arrives without the “may or may not be relevant” advisory framing.
9. Hacker News: How big is your claude.md file?¹ — Practitioner size contest (845 lines vs. a 60–120-line sweet spot) showing length alone doesn’t predict adherence.

Agentic AI Cut the Cost of Building, Not the Cost of Owning

Price the decision on what you’ll own for five years, not on what it costs to ship the first version. The demo that an agent scaffolds in an afternoon is the cheapest moment in the software’s whole life — everything after it is the part you’re actually signing up for.

The distinction is not rhetorical. It is the explicit thesis of the current build-vs-buy literature:

“AI has dramatically reduced the cost of creating software, but it hasn’t eliminated the cost of owning software.”

AI makes custom software cheaper. It does not make custom software free — it moves the bill from the build line, where everyone was looking, to the ownership line, where almost nobody is. The buy option was never priced on how hard the thing was to write; it was priced on the vendor absorbing every patch, migration, and compliance change for the life of the product. Cheapening the build changes one input to a decision whose dominant term was always ownership.

So the build-vs-buy question isn’t about capability anymore — an agent can plausibly build almost anything you’d have bought. It’s about which maintenance tail you’d rather own. That reframe runs through every gate below.

Stop Asking “Can We Afford to Build It?” — Ask “Can We Afford to Own It?”

Before you greenlight a build, multiply the first-build estimate by the maintenance tail — because most of a system’s lifecycle cost lands after the first version ships, and most of that is new work, not warranty repair.

Take the concrete decision this whole post is about: a mid-size regulated company deciding whether to build an internal claims-triage workflow tool — the agentic thing that reads an inbound claim, applies the firm’s triage rules, and routes it — versus buy the SaaS equivalent. Suppose an agent-augmented team estimates the first working version at three months. Run the O’Reilly 60/60 numbers on that estimate.

“Fully 60% of the life cycle costs of software systems come from maintenance, with a relatively measly 40% coming from development.”

If the three-month first build is the 40%, the maintenance that follows is roughly 1.5x that again over the system’s life — and it does not stop. The second 60 is the part that surprises the people who budgeted only the first:

“During maintenance, 60% of the costs on average relate to user-generated enhancements (changing requirements), 23% to migration activities, and 17% to bug fixes.”

Read that breakdown carefully, because it kills the intuition that “the code works, so we’re mostly done.” Only 17% of maintenance is fixing what’s broken. The other 83% is the claims-triage tool absorbing changing requirements — new claim types, a new regulator rule, a payer format that changed — and migrating onto infrastructure that moves underneath it. That work exists whether or not the code was clean, and it is the work an agent cannot pre-pay for you. The sizing question for the claims-triage decision is therefore: not “can we build the first version cheaply?” but “do we want to staff the second 60% of its lifecycle for the next five years?”

Treat AI-Built Code as Maintenance Debt Until Proven Otherwise

Assume AI-generated code carries above-baseline churn and duplication, and budget the review-and-refactor tax before you ship it, not after. The maintenance tail from the last section didn’t just stay heavy in the agentic era — the evidence says AI-assisted code makes it heavier, which is exactly backwards from the “cheaper to build” story.

The measurable signature is duplication crowding out reuse. GitClear’s analysis of changed lines across large enterprise repositories found the trend inverting:

“the percentage of changed code lines (associated with refactoring) sunk from 25% of changed lines in 2021, to less than 10% in 2024, while lines classified as ‘copy/pasted’ (cloned) rose from 8.3% to 12.3%”

Refactoring more than halved as a share of changes while cloning climbed — the maintenance signature of a codebase that grows by pasting, not by consolidating. Duplicated logic is exactly the kind of thing that turns the O’Reilly “changing requirements” 60% into a multi-file hunt: change the triage rule once and you find you’ve shipped it in four places.

Duplication and Reuse Have Inverted

When you see AI raise your line count, check your clone rate before you celebrate velocity. More lines from an agent is the headline number; the clone rate is the one that predicts your maintenance bill — volume up, reuse down. On the claims-triage build, that is the difference between one triage-rules module and the same rule copy-pasted across every route handler the agent touched.

Individual Speed Is Not System Throughput

Don’t read a copilot’s individual speedup as a delivery win; measure stability and throughput, which move the other way. The DORA 2024 research is blunt about the split:

“AI adoption significantly increases individual productivity, flow, and job satisfaction. However, it also negatively impacts software delivery stability and throughput”

The magnitudes make it a rule you can apply, not just a caution. When you see AI adoption rise on a team, expect delivery to move the wrong way:

• When AI adoption rises ~25%, expect throughput to fall ~1.5% — Rachel Stephens (RedMonk): DORA Report 2024 — A Look at Throughput and Stability⁵.
• When AI adoption rises ~25%, expect delivery stability to fall ~7.2% — the larger of the two moves, and stability is the one that costs you at 2 a.m. (RedMonk⁵).

Neither number is catastrophic on its own. Together they say the thing you have to internalize before you build: the individual keystroke got faster, and the system got slightly worse at shipping and staying up. Budget the review-and-refactor tax on the claims-triage code up front, because the velocity you feel at the keyboard is not the throughput you’ll measure at delivery.

Differentiation Is Necessary but Not Sufficient — Add the Verification Gate

Author’s judgment. The gate below — build only when the requirement is both genuinely differentiating and cheaply, continuously verifiable — is my synthesis, not a claim any single source states. It follows from three sourced premises: Fowler’s strategic/utility split (differentiation is what earns a build at all), the O’Reilly/GitClear/DORA maintenance-tail evidence (ownership is where you pay, and AI-built code pays more), and the MIT NANDA base rate later in this post (unverified internal builds mostly fail).

Start with differentiation, because it’s the classic gate and it’s necessary. Martin Fowler’s strategic/utility split is the cleanest statement of it:

“for a strategic function you don’t want the same software as your competitors because that would cripple your ability to differentiate.”

Now classify the claims-triage tool on that axis, and notice it splits — the tool is not one thing:

Utility (buy or rent it): the queue plumbing, the message bus, the retry logic, the model-hosting runtime. Every competitor has the same need and would happily run the same software. Owning it differentiates you from no one and adds maintenance tail for nothing.

Strategic (candidate to build): the triage rules — the business logic that decides how your firm scores and routes your claims. That is the part competitors can’t hand you, and the part Fowler says you don’t want to share.

But differentiation alone doesn’t earn a build, and this is where the classic framework is now incomplete. In the agentic era the build got cheap enough that “it’s strategic” will greenlight far too many projects — and the maintenance-tail and duplication evidence above says most of them will quietly rot. Fowler himself flags that the line moves:

“This is not a static dichotomy. Business activities that are strategic can become a utility as time passes.”

So differentiation is the entry gate, not the decision. The second gate — can you verify it cheaply and continuously? — is the one the next section makes concrete. Something scoped small enough to test in isolation is the operational meaning of “continuously verifiable” (see how to size tasks for AI coding agents).

Build When It’s Business-Specific AND You Can Verify It Cheaply and Continuously

Build only where the requirement is genuinely yours AND tests, logs, or metrics can confirm it’s still correct on every change. If you can’t verify it continuously, buying is cheaper than owning it blind — because “owning it blind” is just the maintenance tail with no early-warning system attached.

Run the two-part gate as a checklist against the claims-triage rules — the strategic slice the last section isolated:

1. Is it genuinely business-specific? The triage rules encode how your firm decides claims. Joel Spolsky’s rule applies directly: “If it’s a core business function — do it yourself, no matter what.” (Joel Spolsky: In Defense of Not-Invented-Here Syndrome⁷). Pass.
2. Can you write an acceptance test for “correct”? Each triage rule maps to observable outcomes — this claim class routes here, that one escalates. If you can express “done” as assertions, you can build it. (Pass.)
3. Can that test run on every change, cheaply? The rules become a test suite the agent — and CI — re-runs on every edit. A rule regression fails a build, not a customer. (Pass.)
4. Would you catch a silent regression within a day? If the answer is “only when a claim is misrouted in production,” you have differentiation without verification — do not build it. (For the rules: pass, via the suite. For an unobservable ML scoring black box: this is where a candidate fails.)

All four clear for the triage rules, and only the triage rules — the surrounding platform does not, which the next section handles. This is why the build case is newly plausible at all: agents make bounded, verifiable work cheap to attempt. Simon Willison names the unlock precisely:

“it’s not about getting work done faster, it’s about being able to ship projects that I wouldn’t have been able to justify spending time on at all.”

That is the opportunity and the trap in one line. Agents let you justify builds you’d have skipped — which is genuinely good when the thing is verifiable, and a slow-motion maintenance disaster when it isn’t. The verification gate is what separates the two. The concrete mechanism for it — acceptance criteria plus verification commands the agent runs itself — is exactly what a well-formed agent task spec already encodes.

Buy When Reliability, Uptime, and Compliance Are the Actual Product

When the deliverable is really audit trails, uptime, and regulatory documentation, buy it — you’re signing up to own a multi-year compliance surface, not a feature. The claims-triage tool lives in a regulated shop, so the platform the rules sit in is not more business logic. It’s a regulatory obligation wearing a software costume. Same tool, two slices, opposite verdicts:

Klotz’s arXiv taxonomy names exactly this category and gives the verdict:

“Mission-critical systems of record: Retain Buy as the primary option. Consider Make selectively for peripheral modules, extensions, or integration layers where the core system’s integrity is not at risk.”

The triage rules are Klotz’s “differentiating custom application” — high specificity, Make. The platform around them — model lifecycle management, sandboxing, audit-evidence capture, the regulatory surface — is a “regulated standard application,” where compliance is the product and Buy is the answer. You build the rules and buy the compliance chassis they run in.

The cost model shows why you don’t want to own the chassis. GitLab’s build economics for a regulated agentic platform:

“For a team of roughly 200 developers, an internal build typically costs around $1.4M in year one, requires 2–3 dedicated FTEs to maintain, and takes 12–18 months to reach a first real use case.”

(GitLab sells the bought alternative, so treat the dollar figure as directional, not gospel.) Even directionally, the 12–18 months is the tell: that is time your differentiating triage rules are not shipping because your engineers are building an audit-log system a vendor already sells. Ross names the opportunity cost exactly — “Every engineer building the platform is an engineer not modernizing a legacy pipeline, remediating security debt, or accelerating a critical delivery program” (Bryan Ross (GitLab)¹⁰). Buy the chassis. Spend the engineers on the rules.

Run the Base Rate Before You Build — Naive Internal Builds Mostly Fail

Before you trust “agentic AI makes this easy to build,” price in the base rate — internal builds succeed about a third as often as bought tools, so demand a verification story that beats it. The demo that convinced you is not evidence about the tail; it’s evidence about the easiest 40%.

The base rate is stark. MIT’s NANDA research, reported by Fortune, splits outcomes by approach:

“Purchasing AI tools from specialized vendors and building partnerships succeed about 67% of the time, while internal builds succeed only one-third as often.”

One-third as often. Stress-test the claims-triage build against that number: if bought tools land 67% of the time and the naive internal build lands around a third of that, your build has to clear a bar the market default clears twice as easily. The verification gate from two sections back is precisely the thing that can move you above the base rate — it’s the difference between “we built it” and “we can tell, on every change, whether it still works.”

And the reason “cheap to build” isn’t “likely to survive contact with your codebase” is that the productivity story inverts on exactly the systems you’d own. METR measured it:

“When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts.”

Slower, on work developers were sure got faster. Simon Willison pinpoints why it matters for a build-vs-buy call:

“The factor that stands out most to me is that these developers were all working in repositories they have a deep understanding of already, presumably on non-trivial issues since any trivial issues are likely to have been resolved in the past.”

That is the claims-triage tool a year after you ship it — a mature codebase your team knows deeply, full of non-trivial issues. The AI edge that made the first build feel cheap is smallest exactly there. Which is the whole argument, closed: the demo is cheap, the tail is expensive, and the tail is where you actually live.

Decide with the Ownership-Cost Flowchart

Walk any candidate through the gates in order — ownership cost, differentiation, continuous verification, compliance-as-product, base rate — and build only what clears all five. The order matters: cheap gates that eliminate candidates come first, so you don’t run the expensive analysis on things you were never going to build. Run the claims-triage decision through it end to end.

BUILD-vs-BUY: the ownership-cost flowchart
───────────────────────────────────────────

1. OWNERSHIP COST — Multiply the first-build estimate by the maintenance tail.
   (~1.5x lifecycle beyond the build; 83% of it is new work, not bug-fixing.)
   Would you staff the second 60% for five years?
      NO  → BUY. Stop.
      YES → ↓
                                    [claims-triage: yes, IF it's the rules only]

2. DIFFERENTIATION — Is this the strategic slice competitors can't hand you?
      NO (it's utility: queues, runtime, plumbing) → BUY / RENT that layer. Stop.
      YES → ↓
                       [triage RULES: strategic → continue │ PLATFORM: utility → buy]

3. CONTINUOUS VERIFICATION — Can a test/log/metric confirm it's still correct
   on EVERY change, cheaply — and would you catch a silent regression within a day?
      NO  → BUY. Differentiation without verification is owning it blind. Stop.
      YES → ↓
                                    [triage rules: yes, via a rules test suite]

4. COMPLIANCE-AS-PRODUCT — Is the real deliverable audit trails, uptime,
   regulatory evidence (a "regulated standard application")?
      YES → BUY the platform; Make only peripheral, low-risk modules. 
      NO  → ↓
                       [triage PLATFORM: yes → BUY │ triage RULES: no → continue]

5. BASE RATE — Naive internal builds succeed ~1/3 as often as bought tools,
   and AI's edge fades on codebases you own and know.
   Does your verification story (gate 3) beat that base rate?
      NO  → BUY. Your build is below the market default. Stop.
      YES → BUILD it.
                                    [triage rules: verifiable → BUILD]

───────────────────────────────────────────
VERDICT: Build the strategic, continuously-verifiable triage RULES.
         Buy the regulated PLATFORM they run in.

The claims-triage decision resolves the same way every published gate predicted it would. The rules clear ownership cost (verifiable, so the tail is bounded), differentiation (they’re your business logic), continuous verification (a rules suite), compliance-as-product (they’re not the compliance surface — the platform is), and the base rate (the verification story beats one-third). The platform fails at differentiation and compliance and gets bought. That is the answer agentic AI actually changed: not “build more,” but “build the small strategic core you can verify, and buy the expensive tail you’d only own blind.”

References

Research and Data

1. GitClear: AI Copilot Code Quality — 2025 Look Back at 12 Months of Data³ — Refactoring fell from 25% of changed lines (2021) to under 10% (2024) while cloned lines rose 8.3%→12.3%; backs the AI-code-as-maintenance-debt and duplication-inversion sections.
2. Rachel Stephens (RedMonk): DORA Report 2024 — A Look at Throughput and Stability⁵ — Verbatim magnitudes: +25% AI adoption → throughput −1.5%, stability −7.2%; backs “individual speed is not system throughput.”
3. DORA (Google Cloud): Accelerate State of DevOps Report 2024⁴ — AI adoption raises individual productivity but hurts delivery stability and throughput; backs the same section.
4. Sheryl Estrada (Fortune / MIT NANDA): MIT report — 95% of generative AI pilots at companies are failing¹¹ — Bought tools/partnerships succeed ~67% of the time; internal builds succeed one-third as often. Backs the base-rate section.
5. METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity¹² — AI made experienced devs 19% slower on their own repos; backs “cheap to build ≠ likely to survive your codebase.”
6. David Klotz (arXiv): The Buy-or-Build Decision, Revisited⁹ — Taxonomy placing regulated and mission-critical systems in the Buy column; backs “buy when compliance is the product.”

Practitioner Guidance

7. Matt Paige (HatchWorks): The Build vs Buy Framework in the Age of AI¹ — States the build-cheap / own-expensive thesis verbatim; backs the opening reframe.
8. David Wood (O’Reilly): The 60/60 Rule² — 60% of lifecycle cost is maintenance, and 60% of that is changing-requirements enhancement (17% is bug fixes); backs “can we afford to own it?”
9. Martin Fowler: Utility Vs Strategic Dichotomy⁶ — Strategic functions must not run the same software as competitors, and the classification shifts over time; backs the differentiation gate.
10. Joel Spolsky: In Defense of Not-Invented-Here Syndrome⁷ — Do your core business function in-house no matter what; backs “build when it’s business-specific.”
11. Simon Willison: Here’s how I use LLMs to help me write code⁸ — LLMs unlock projects you couldn’t justify before; backs the marginal-build unlock behind the verification gate.
12. Simon Willison: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity¹³ — The slowdown lands hardest in repositories developers already know deeply; backs the base-rate section.
13. Bryan Ross (GitLab): The real cost of build vs. buy for agentic AI in regulated industries¹⁰ — ~$1.4M year-one build, 2–3 FTEs, 12–18 months, plus the opportunity-cost framing; backs “buy the regulated platform.” Vendor-authored; dollar figures treated directionally.
14. Pat Brans (CIO.com): Your next big AI decision isn’t build vs. buy — It’s how to combine the two¹⁴ — The hybrid “combine the two” reframe; context for why this post gates on verification rather than restating the hybrid consensus.
15. Digital Applied Team: Build vs Buy — The 2026 Case for Custom AI Tools¹⁵ — Reframes the 2026 question around vendor lock-in; context for the differentiation-and-verification lens this post takes instead.

Author’s Judgment (not directly sourced)

The following claim is my own synthesis. It follows logically from the sourced material above, but no source states it directly:

• “The two-part verification gate” (build only when the requirement is both differentiating and cheaply, continuously verifiable) — follows from Fowler’s strategic/utility split (differentiation earns a build), the O’Reilly/GitClear/DORA maintenance-tail evidence (ownership is where you pay, and AI-built code pays more), and the MIT NANDA base rate (unverified internal builds mostly fail). No single source states the combined gate.

Ask Two Questions Before You Write the Prompt: Can You Verify It, and Can You Undo It?

Score the task before you reach for the agent, not the agent before you reach for the task. The tool listicles rank context windows and token efficiency; none of them name the two axes that actually decide whether delegation pays off — how expensive it is to check the result, and how expensive it is to reverse a wrong one. If either axis is bad, the task stays in your hands, no matter which agent tops this quarter’s benchmark.

This is the same discipline Anthropic leads with in its own agent-building guidance: start with the simplest thing that works, and add autonomy only when the task earns it.

“When building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all.”

“Not building agentic systems at all” is the option the rankings never present, because a ranking exists to sell you a tool. Those two axes are the reader’s whole apparatus for the rest of this post: every section that follows is one lens on where a task lands.

Hold one incident in mind as the case study, because the later sections dissect it from every angle. A user asked an agent to “clean up old branches.” It listed the remote branches, pattern-matched what looked old, and issued a delete that would have destroyed history (Anthropic: How we built Claude Code auto mode²). Vague goal, no cheap correctness check, irreversible blast radius — one command that fails both axes at once. Keep it in view.

Hand Agents Bounded, Verifiable Work With a Tight Feedback Loop — That’s Where They Fly

Delegate the tasks that arrive with a pass/fail check the agent can run without you. That closed loop — write code, run the check, read the result, iterate — is the exact mechanism agents exploit, and coding is the archetypal domain where it exists, because a test suite turns “looks done” into “the check passed.”

Sean Goedecke, a staff engineer, describes where agents win for him now:

“I now use LLMs to produce entire PRs in areas I’m familiar with”

Entire PRs, in areas he already knows — the two qualifiers are the whole point. In a familiar area he can write the verifying check cheaply and read the diff fast, and on bug investigation he reports the agent is “able to correctly diagnose 80% of issues on its own” (Sean Goedecke: How I use LLMs as a staff engineer in 2026³). The agent flies because the feedback loop is tight and the correctness signal is cheap, not because the model is magic.

The enabler is the test suite. Simon Willison makes the dependency explicit: the closed loop is only as good as the check that closes it.

“If your project has a robust, comprehensive and stable test suite agentic coding tools can fly with it.”

This is not marketing enthusiasm — it is a mechanical claim about verifiability. Anthropic scopes it precisely: “Code solutions are verifiable through automated tests; Agents can iterate on solutions using test results as feedback” (Anthropic: Building effective agents¹). The test result is the feedback. No test, no loop, no flight. Once a task passes this go/no-go bar, the next move is writing the spec well — the anatomy of a perfect AI agent task covers the elements that make a bounded task land on the first try.

Give the Agent the Verification Command — Don’t Be the Verification Command Yourself

The failure mode inside the fit case is subtle: you hand over bounded work but keep the correctness check in your own head, so every mistake waits for you to notice it. The move is to externalize the check into something the agent can run.

Bad: “Add E.164 validation to UserService.ValidatePhone, then tell me when it’s done.” (Done how? You are now the loop — you read the diff, you decide if it’s right, and the agent’s confidence is your only signal.)

Good: “Add E.164 validation to UserService.ValidatePhone. Run go test ./internal/user/... -run TestValidatePhone and show me the output before declaring it done.” (Now the check runs itself and the agent can’t call “looks done” the finish line.)

The Claude Code docs draw the line at the evidence:

“Have Claude show evidence rather than asserting success: the test output, the command it ran and what it returned, or a screenshot of the result.”

Evidence over assertion. If the task has no runnable check, you haven’t given the agent bounded work — you’ve given yourself an unreviewed diff.

Refuse Long-Horizon, Unbounded Tasks — Agent Reliability Falls Off a Measurable Cliff

Estimate how long the task would take a competent human before you delegate it — because agent success does not decay gently with task length, it collapses. METR measured exactly where. The success rate is near-total on the short tasks and near-zero on the long ones:

“current models have almost 100% success rate on tasks taking humans less than 4 minutes, but succeed <10% of the time on tasks taking more than around 4 hours”

That is the cliff, and it hands you a portable rule keyed to human time — the one estimate you can make before writing a line of spec:

The reason the rule works long-term is that the horizon is moving, but slowly and predictably. METR’s methodology paper puts a number on the trend: “frontier AI time horizon has been doubling approximately every seven months since 2019” (Kwa, West, Becker et al. (METR): Measuring AI Ability to Complete Long Software Tasks⁷). Doubling every seven months means the four-hour ceiling rises over time — but on any given day it is a hard, measurable ceiling, and a multi-hour slog sits well past it. Decomposing a too-big task is its own skill; how to size tasks for AI coding agents walks the seams. The point here is upstream of sizing: if the human estimate says “hours,” the task is not one task, and handing it over whole is betting against a <10% success rate.

When Verification Costs More Than Writing the Code, Do It Yourself

Compare the cost of checking the agent’s output against the cost of writing it yourself — and when checking wins, write the code. This is the axis the reliability cliff misses. A task can sit comfortably under the four-hour horizon and still be a net loss, because the expensive part is not producing the diff, it is proving the diff is correct.

The proving is the constraint now, not the typing. Addy Osmani frames the shift in one line:

“AI writes faster. Humans still have to prove it works.”

And the review bill, when it comes due, is heavier than the writing it replaced. Osmani cites survey data that both halves of the problem are real — most developers skip the check, and those who do the check find it harder than reviewing human code:

“only 48% of developers consistently check AI-assisted code before committing it, even though 38% find that reviewing AI-generated logic actually requires more effort than reviewing human-written code.”

The trap hides in that 38%. Reviewing AI logic costs more per line, so a task where you’d have to reconstruct the full context to verify the output is a task where verification can quietly exceed authorship. Osmani names where the cost concentrates: “AI gets you 80% to an MVP; the last 20% requires patience, learning deeply or hiring engineers” (Addy Osmani: The 80% Problem in Agentic Coding⁹). That last mile is verification-heavy by nature, and it is where the agent’s speed advantage inverts into a review tax.

Run “clean up old branches” through this axis alone and it fails before you consider anything else. There is no cheap check that the branches the agent pattern-matched are the disposable ones — confirming that costs more attention than deleting the right branches by hand. When the verification is harder than the work, delegating it is not a shortcut. It is a detour with a review bill at the end.

Skip Agents When the Goal Is Vague, Taste-Heavy, or the Context Lives in Your Head

If you can’t write a verifiable “done,” don’t hand the task to an agent — that is where measured productivity goes negative, not just flat. The failure is not that the agent works slowly on ambiguous goals. It is that it works confidently in the wrong direction, and you pay to discover it.

METR’s field study is the hard evidence, and it is counterintuitive enough to sit with. On real, high-context open-source work:

“When developers are allowed to use AI tools, they take 19% longer to complete issues”

Nineteen percent longer — on their own repos, where the context lived in their heads and nowhere the agent could read it. And the perception gap is the part that should scare you: the same developers “still believed AI had sped them up by 20%” after the study, having expected 24% going in (METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity¹⁰). Gut feel said faster; the stopwatch said slower — so the skip decision has to be made by rule, not by feel. Skip the agent when the task is any of these:

• A judgment-writing task — ADRs, design docs, Slack messages. Goedecke is blunt: “I still don’t use LLMs to write Slack messages, ADRs, issues and so forth” (Sean Goedecke: How I use LLMs as a staff engineer in 2026³).
• A taste-heavy change — look-and-feel UI, tone, the “core idea” of a change that only reads right when a human wrote it.
• A vague cleanup — “clean this up,” “make it better,” anything where “done” is in your head and not on the page.
• Business logic whose why only you hold — where the correct behavior depends on context that was never written down.

The line under all four is maintenance. Goedecke’s own boundary: “LLMs excel at writing code that works that doesn’t have to be maintained” (Sean Goedecke: How I use LLMs as a staff engineer¹¹) — and it’s “rare that I let Copilot produce business logic for me” (Sean Goedecke: How I use LLMs as a staff engineer¹¹). Throwaway and verifiable, yes. Load-bearing and judgment-bound, no.

“Clean up old branches” is the checklist’s first two items fused into one instruction. It is a vague cleanup and a taste call about what counts as “old,” with the real intent — probably “delete my merged local branches” — living entirely in the user’s head. The agent had to guess “old,” and it guessed destructively. A goal you can’t verify is a goal the agent will complete in a direction you didn’t mean.

Size the Blast Radius: Never Delegate an Irreversible Action You Can’t Cheaply Catch

Before you delegate, ask what a wrong result you don’t catch actually costs — and gate any irreversible, hard-to-detect action behind your own hands, however verifiable the task looks. This is the standalone axis, orthogonal to the other three. A task can be short, cheap to verify, and clearly specified and still fail here, because the question is not “will the agent get it right” but “what happens the one time it doesn’t.”

The reassuring part is that irreversible actions are rare. Anthropic’s analysis of real agent traffic puts a number on it:

“80% of tool calls come from agents that appear to have at least one kind of safeguard (like restricted permissions or human approval requirements), 73% appear to have a human in the loop in some way, and only 0.8% of actions appear to be irreversible”

Zero point eight percent. But rarity is exactly why the axis needs its own gate — the base rate lulls you, and Anthropic is direct about the asymmetry: “while these higher-risk actions are rare as a share of overall traffic, the consequences of a single error can still be significant” (Anthropic: Measuring AI agent autonomy in practice¹²). One irreversible mistake in a thousand safe actions is not a 0.1% problem if the one mistake destroys production data.

“Clean up old branches” is this axis’s home, the worked example the whole taxonomy points at. The agent’s delete would force-push away history — precisely the destructive-action class Anthropic names: “Cause irreversible loss by force-pushing over history, mass-deleting cloud storage, or sending internal data externally” (Anthropic: How we built Claude Code auto mode²). That is why the incident fails even a reader who fixes every other axis: suppose the goal were crisp and the check were cheap — the action is still irreversible and destructive, and a wrong result you don’t catch in the moment is gone. So the blast-radius gate overrides the other three: when the undo is expensive or impossible, the answer is your own hands, no matter how clean the task scored elsewhere.

Run the Task Through Both Axes Before You Open a Prompt

Score every candidate task on verification cost and reversibility first; only the cheap-to-verify, easy-to-undo quadrant goes to an agent unsupervised. The five sections above collapse into one ordered pass — run a task through the gates in sequence, and the first failure is your answer.

                THE GO / NO-GO PASS
                run before you write the prompt

1. Is there a check the agent can run itself?        (bounded, verifiable work)
   test suite / build / lint / diff-against-fixture
      NO  ─────────────────────────────►  KEEP IT. No closed loop, no delegation.
      YES ▼

2. How long would this take a competent human?       (the reliability cliff)
      > a few hours ──────────────────►  DECOMPOSE into sub-hour verifiable pieces,
                                          or keep it. Success is <10% past 4 hours.
      ≤ a few hours ▼

3. Would checking the output cost more than           (the verification tax)
   writing it yourself?
      YES ─────────────────────────────►  WRITE IT. Review harder than authorship
                                          is a net loss, not a shortcut.
      NO  ▼

4. Can you state "done" as something verifiable?      (vague / taste / hidden context)
   not taste, not judgment, not context-in-your-head
      NO  ─────────────────────────────►  KEEP IT. Measured productivity goes
                                          negative on high-context work (19% slower).
      YES ▼

5. If a wrong result slips through uncaught,           (size the blast radius)
   is the action reversible AND cheap to detect?
      NO  ─────────────────────────────►  YOUR HANDS. Irreversible + hard-to-catch
                                          overrides every gate above.
      YES ▼

   ►  DELEGATE unsupervised. Cheap-to-verify, easy-to-undo quadrant.

Now run “clean up old branches” back through it, because a decision tool that only passes easy cases is worthless. Gate 1: there is no cheap check that the pattern-matched branches are the disposable ones — it stumbles immediately. Even granting a check, gate 4: “old” is a taste call with the real intent in the user’s head — it fails again. And gate 5: the delete force-pushes away history — irreversible, hard to catch, straight to your hands. The incident that opened this post fails three gates independently, and any one of them would have caught it before the prompt was ever written.

That is the whole apparatus. Not “which agent is best” — the question the rankings answer and the reader never asked — but “does this task belong to an agent at all.” And the honest coda is that even a task that clears all five gates still earns a look at the diff. Goedecke, who is bullish on agents, still reports: “For difficult tasks, I’ll often reject five or six (or more!) agent attempts before accepting one as good enough to work with, or giving up and making the change by hand” (Sean Goedecke: How I use LLMs as a staff engineer in 2026³). The gates decide what you delegate. Judgment still decides what you accept.

References

Research and Data

1. METR: Measuring AI Ability to Complete Long Tasks⁶ — The reliability cliff: near-100% success under 4 minutes, <10% past ~4 hours; ~7-month doubling of the task horizon.
2. Kwa, West, Becker et al. (METR): Measuring AI Ability to Complete Long Software Tasks⁷ — The peer-reviewable backbone for the horizon trend: doubling approximately every seven months since 2019.
3. METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity¹⁰ — On real high-context repos, developers took 19% longer with AI while believing they were 20% faster.
4. Addy Osmani: The 80% Problem in Agentic Coding⁹ — 48% don’t consistently check AI code; 38% find reviewing it harder than human code; the last-20% tax.
5. Anthropic: Measuring AI agent autonomy in practice¹² — Only 0.8% of real agent actions are irreversible, but the consequences of a single error can still be significant.

Practitioner Guidance

6. Anthropic: Building effective agents¹ — Start simple, up to and including not building agentic systems at all; code is verifiable through automated tests.
7. Simon Willison: Vibe engineering⁴ — A strong, stable test suite is what lets agentic coding tools fly.
8. Claude Code Docs: Best practices for Claude Code⁵ — Give the agent a check it can run; have it show evidence rather than assert success.
9. Sean Goedecke: How I use LLMs as a staff engineer in 2026³ — Entire PRs in familiar areas, 80% of bugs diagnosed, but ADRs stay human and hard tasks get five or six rejected attempts.
10. Sean Goedecke: How I use LLMs as a staff engineer¹¹ — Agents excel at code that doesn’t have to be maintained; rarely for business logic.
11. Addy Osmani: Code Review in the Age of AI⁸ — AI writes faster, but a human still has to prove it works; review is the moved bottleneck.
12. Anthropic: How we built Claude Code auto mode² — The “clean up old branches” incident and the destructive-action taxonomy (force-push, mass-delete, exfiltrate).

The Real Constraint: Context, Not Lines of Code

People instinctively think about task size in terms of lines of code or number of files changed. Those are secondary proxies. The actual limiter is how much context the agent must consume — reading files, exploring the codebase, running commands, processing outputs — before it can do the work.

This matters because model performance degrades as context fills. Chroma’s research report on context rot, which measured 18 LLMs, found that models do not attend to their context uniformly — performance grows increasingly unreliable as input length grows (Chroma: Context Rot — Hong et al., 2025¹, cited via Factory.ai: The Context Window Problem²). This isn’t a cliff; it’s a slope. Instructions from early in the conversation get progressively less likely to be followed as the window fills.

Anthropic’s own engineering team reinforces this: even as context windows get larger, treating context as a precious, finite resource will remain central to building reliable agents. Bigger windows don’t eliminate the problem — they just move the degradation curve further out (Anthropic: Effective Context Engineering for AI Agents³).

The sizing question is therefore: can this task be completed before the agent’s reasoning quality degrades?

Heuristic 1: One Coherent Thing Per Session

The Claude Code documentation calls this out directly: the most effective context management strategy is to do one thing per session. Their example: instead of “build the notification feature,” break it into “add the notifications database table” and start a fresh context for each step (Developer Toolkit: Context Windows⁴).

The key word is “coherent.” This doesn’t mean “one tiny thing” — it means one thing with a single logical boundary that can be verified in isolation. A migration. An endpoint. A service method with its tests. Not artificially small, but naturally scoped.

The Claude Code docs also describe the antipattern to watch for — the “kitchen sink session” — where you start with one task, ask something unrelated, then go back to the first task. Context fills with irrelevant information and quality drops (Claude Code Docs: Best Practices⁵).

Heuristic 2: The One-Sentence Diff Test

The Claude Code docs offer a useful litmus test for whether a task needs a plan or can be executed directly: “If you could describe the diff in one sentence, skip the plan” (Claude Code Docs: Best Practices⁵).

Flip this into a sizing tool: if you can’t describe the expected diff in one sentence, the task is either too big or needs decomposition. “Add a nullable phone_number column to the users table with an up and down migration” is one sentence — that’s a well-sized task. “Add phone number support across the full stack” is not one sentence — that’s multiple tasks.

Heuristic 3: Count Files Read, Not Files Changed

Author’s judgment. This is my own inference, not a direct claim from any source. The reasoning chain follows from sourced premises.

Most people estimate task complexity by how many files will change. But the bigger cost is how many files the agent needs to read to understand enough context to make those changes safely.

The underlying mechanics are real: each file read, grep, and bash execution consumes context and, depending on the tool, counts as a separate API call. If the agent reads 20 files to understand the codebase before writing a single line, that’s significant context burn before the real work begins (Claude Code Guides: Rate Limits⁶).

Practical implication: If a task requires the agent to understand too much of the system to make a safe change, it’s either too big or you haven’t provided enough upfront context (relevant files, reference implementations, architectural notes) to shrink the exploration phase. This is where a well-written task spec pays for itself — you’re effectively pre-loading the context the agent would otherwise have to discover on its own.

Heuristic 4: Independent Verifiability

A well-sized task produces output that can be verified without needing the next task to be done first. You should be able to run tests, lint, or hit an endpoint to confirm the task worked in isolation.

This maps directly to milestones. Each one should be a checkpoint where you can say “this works” or “this doesn’t” before moving on. If you can’t verify a task independently, it’s either too small (a fragment of a meaningful change) or too entangled with other work (and should be restructured).

Google Cloud’s best practices for AI coding assistants recommend creating a set of tests that will determine if the generated code works based on your requirements — essentially making verification part of the task definition, not an afterthought (Google Cloud: Five Best Practices for AI Coding Assistants⁷).

Heuristic 5: Layer Boundaries Are Natural Decomposition Points

Author’s judgment. Derived from the “one coherent thing” principle above, not stated directly in any source.

If a task requires changes across multiple architectural layers simultaneously — a new database column and a new API endpoint and new frontend UI and updated docs — that’s not one task. Each layer boundary is a natural split point because each layer can typically be verified independently.

The example from the companion guide (The Anatomy of a Perfect AI Agent Task) splits work this way:

1. Migration — Add the column, verify migrate up/down.
2. Model + generated code — Update the struct and queries, verify no diff.
3. Service + validation — Add business logic, unit test it.
4. Handler + integration — Wire up the endpoint, integration test the full flow.

Each milestone is independently verifiable and produces a clean commit. If step 3 fails, you don’t lose the work from steps 1 and 2.

This isn’t a universal law — sometimes a change is so small that splitting it across layers would create more overhead than it saves. But for anything non-trivial, layer boundaries are the most reliable seams for decomposition.

Heuristic 6: Don’t Over-Decompose

Author’s judgment. The specific framing is my own, though it’s consistent with the general principle of matching effort to task complexity found in multiple sources.

There’s a floor to useful task size. If a task is so trivial that you could do it faster manually than writing the task spec, starting a session, and reviewing the output — it’s too granular. “Add a column to the struct” as a standalone task doesn’t give the agent any meaningful opportunity to verify its work and you’re paying the overhead of a fresh context for something trivial.

The MindStudio guide on session limits makes a similar point about effort levels: max effort on a trivial task wastes context, while low effort on a complex refactor produces incomplete output. Match the investment to the task (MindStudio: Claude Session Limits⁸).

Bundle trivial changes with the next logical step. “Add the column to the struct and update the queries and regenerate the generated code” is a natural unit — it’s still one coherent thing (the model layer), but it has enough substance for the agent to verify.

The Practical Sweet Spot

Note: The specific numbers below are my translation of PR sizing research into AI agent task heuristics. The underlying research is well-established; the mapping to “files read” and “files changed” is my own approximation for a typical Go codebase and will vary by language and architecture.

What the research says about reviewable units of work

The PR sizing research is extensive and converges on consistent numbers:

• Google’s engineering research found that review quality drops significantly for PRs exceeding 200 lines of changed code, and review time increases non-linearly with size. They recommend keeping PRs under 200 lines (EM Tools: Pull Request Size⁹).
• An analysis of 50,000+ pull requests across 200+ teams found that PRs with 200–400 lines changed have 40% fewer defects than larger PRs, PRs over 1,000 lines have 70% lower defect detection rates, and each additional 100 lines increases review time by 25 minutes (Propel: Impact of PR Size on Code Review Quality¹⁰).
• Graphite’s analysis found that 50-line changes are reviewed and merged roughly 40% faster than 250-line changes and are 15% less likely to be reverted (Graphite: The Ideal PR is 50 Lines Long¹¹).
• High-performing teams typically enforce a soft limit of 400 lines and a hard limit of 600 lines, above which the PR must be split (EM Tools⁹, Augment Code: Code Review Best Practices¹²).

My translation for AI agent tasks

Given a ~200 LOC target for the diff:

• Files changed: ~2–5. In a typical Go project following a layered architecture, a 200-line change across a model file, service method, handler, test file, and a migration lands in this range. This will vary by language — a React project with smaller component files might touch more files for fewer total lines.
• Files read: roughly 2× the files changed. To make a safe change to 3 files, the agent typically needs to read those 3 files plus their interfaces, dependencies, and test fixtures. This is a rough multiplier, not a researched number.
• Review time: under 5 minutes. A 200-line diff at a reasonable reading pace takes about 5 minutes to review, which aligns with the research showing that review effectiveness drops sharply beyond the first hour, with most value extracted in the first pass.

These are approximations. The underlying principle is more durable than the specific numbers: size your tasks so the output maps to a single, reviewable pull request that a human can understand quickly.

Sizing Decision Flowchart

When deciding if a task is the right size, ask these questions in order:

Can you describe the expected diff in one sentence? If no → the task needs decomposition.

Can the result be verified independently? If no → you’ve either scoped too small (a fragment) or too entangled (restructure it).

Does it cross multiple architectural layers? If yes → split along layer boundaries unless the total change is trivial.

Does the agent need to read more than ~10 files to understand enough context? If yes → either the task is too broad, or you need to provide more upfront context in the task spec to reduce exploration. (Note: the “~10 files” number is author’s judgment, not a researched threshold.)

Could you do it faster manually than writing the task spec? If yes → bundle it with the next logical step.

References

Research and Data

1. Chroma: Context Rot — Hong et al., 2025¹ — Measured 18 LLMs and found performance degrades non-uniformly as context length grows. Cited via Factory.ai.
2. Google’s PR Sizing Research⁹ — Review quality drops significantly above 200 LOC; recommends keeping PRs under 200 lines.
3. Propel: Impact of PR Size on Code Review Quality¹⁰ — Analysis of 50,000+ PRs: 200–400 line PRs have 40% fewer defects; each additional 100 lines adds ~25 minutes of review time.
4. Graphite: The Ideal PR is 50 Lines Long¹¹ — 50-line changes are merged 40% faster and 15% less likely to be reverted than 250-line changes.

Practitioner Guidance

5. Anthropic: Effective Context Engineering for AI Agents³ — Context is a finite resource regardless of window size; compaction and just-in-time retrieval over pre-loading.
6. Claude Code Docs: Best Practices⁵ — One thing per session, the “kitchen sink” antipattern, and the one-sentence diff test.
7. Factory.ai: The Context Window Problem² — Enterprise codebases exceed context windows; context must be curated like CPU time or memory.
8. Google Cloud: Five Best Practices for AI Coding Assistants⁷ — Test-driven verification as part of task definition.
9. MindStudio: Claude Session Limits⁸ — Match effort level to task complexity; sub-agents for work that exceeds a single session.
10. Augment Code: Code Review Best Practices¹² — Sub-400 LOC PRs and sub-six-hour completion times as scaling benchmarks.

Author’s Judgment (not directly sourced)

The following claims are my own synthesis. They follow logically from the sourced material above, but no source states them directly:

• “Count files read, not files changed” — Inferred from context window mechanics and API call patterns.
• “Layer boundaries are natural decomposition points” — Derived from the “one coherent thing per session” principle.
• “Don’t over-decompose” / the minimum viable task size — Derived from the principle of matching effort to complexity.
• “2–5 files changed / 2× files read” — My approximation for Go codebases based on the ~200 LOC PR sizing research. Will vary by language and architecture.
• "~10 files" exploration threshold — A rough heuristic for when a task requires too much context discovery. Not a researched number.

When to Use This

The seven elements below describe the upper-bound shape of a non-trivial task spec, not a baseline checklist. For trivial work — fixing a typo, renaming a variable, anything where the agent has no real risk of getting it wrong — skip the elaborate spec. (The companion sizing post uses “describable in one sentence” as a sizing test, not a triviality test — well-sized tasks often fit in one sentence yet still warrant a full spec when there are constraints, edge cases, or pitfalls to communicate. The worked example below is one such task.) Even for non-trivial tasks, treat these elements as a maximum rather than a minimum: frontier LLMs reliably follow only ~150–200 instructions before performance degrades, and every irrelevant detail dilutes the signal of the rest (HumanLayer: Writing a Good CLAUDE.md²).

1. State the Goal, Not the Steps

Lead with the outcome you want, not a micro-managed sequence of instructions. Agents perform better when they understand the “why” and can plan their own approach.

Bad: “Open user.go, find the CreateUser function, add a field called PhoneNumber…” Good: “Add phone number support to user registration, including validation, storage, and API response.”

“The best task descriptions share three properties: they state the goal, provide constraints, and define done.”

2. Provide Architectural Context the Agent Can’t Infer

The agent can read your code. What it can’t read is the reasoning behind your architectural decisions, team conventions, or the “why” behind structural choices. Include only what’s not derivable from the codebase itself.

Include things like:

• Why the architecture is shaped a certain way (e.g., “We use the repository pattern to keep DB logic out of handlers”)
• Relevant files and entry points (saves the agent from searching blindly and burning context window)
• Technology choices and versions (e.g., “Go 1.22, sqlc for query generation, chi router”)
• Domain-specific terminology the agent might misinterpret

“Claude already knows what your project is after reading a few files. What it needs is information it can’t derive from reading code.”

That said, there’s a discipline to this — more context is not always better. Research suggests frontier LLMs can reliably follow roughly 150–200 instructions before performance degrades, and broader context-rot studies show models attend to context less reliably as input grows (Chroma: Context Rot — Hong et al., 2025⁴). Every irrelevant detail you add dilutes the signal of the details that actually matter.

“Your CLAUDE.md file should contain as few instructions as possible — ideally only ones which are universally applicable. An LLM will perform better when its context window is full of focused, relevant context compared to when it has a lot of irrelevant context.”

3. Define Explicit Constraints and Non-Goals

This is where most tasks fall apart. Without boundaries, agents will happily refactor your auth layer while you asked them to add a field to a struct.

• Constraints: What rules must be followed (e.g., “Do not change the public API contract,” “Use the existing validate package, do not introduce a new dependency”)
• Non-goals: What is explicitly out of scope (e.g., “Do not modify the frontend,” “Do not refactor existing tests”)

“Without constraints, AI might miss pagination for list APIs, use field injection instead of constructor injection, or not adhere to your project’s package structure.”

4. Provide Concrete Examples and Reference Implementations

One of the highest-leverage things you can do. Point the agent at an existing implementation in your codebase that follows the pattern you want replicated.

• “Follow the same pattern as internal/order/handler.go for the new endpoint.”
• “See migrations/003_add_email.sql for the migration format we use.”

“Include helpful examples for reference. ❌ ‘Implement tests for class ImageProcessor’ → ✅ ‘Implement tests for class ImageProcessor. Check text_processor.py for test organization examples.’”

5. Define “Done” with Acceptance Criteria

If you don’t define what “done” looks like, the agent will decide for you — and you probably won’t agree.

Acceptance criteria should be:

• Observable (can be verified by running something)
• Specific (not “should work correctly”)
• Testable (ideally map to test cases)

“Create a set of tests that will determine if the generated code works based on your requirements.”

6. Include Verification Commands

Tell the agent exactly how to confirm its own work. This is the difference between “I think it works” and “it passes the build.”

• go test ./internal/user/...
• go vet ./...
• golangci-lint run
• curl -X POST localhost:8080/api/v1/users -d '{"phone": "+1234567890"}' | jq .

“Claude Code’s best practices emphasize including Bash commands for verification. This gives Claude persistent context it can’t infer from code alone.”

7. Call Out Edge Cases and Known Pitfalls

You know things about your system the agent doesn’t. If there’s a footgun, flag it. If there’s a non-obvious coupling between modules, say so.

• “The user_id column has a unique constraint — the migration must handle existing duplicates.”
• “The Validate() method is called both at the handler level and inside the repository. Don’t double-validate.”

The Full Example

A non-trivial feature decomposes into a handful of well-sized tasks. Take adding an optional phone number to user registration — accepted on signup, persisted on the user record, and returned by the user API. That feature splits into four tasks, one per architectural layer:

1. Migration — Add a nullable phone_number column with reversible up/down SQL.
2. Model + sqlc — Update the User struct and regenerate sqlc queries.
3. Service + validation — Add ValidatePhone to UserService using validate.PhoneE164, with unit tests.
4. Handler + integration — Wire the field through POST and GET /api/v1/users and add integration tests.

The third is spec’d out in full below as the worked example. It’s the strongest illustration of the seven elements at the right scope: the diff fits in one sentence, it stays inside a single layer, the agent reads ~5 files, the change lands well under the 200 LOC ceiling, and it can be verified independently — passing every gate of the companion sizing post’s decision flowchart.

## Task Spec: Add E.164 phone validation to UserService

### Goal
Phone numbers submitted to user registration must be rejected at the service layer when they aren't valid E.164. This task delivers that check; handler wiring and DB persistence are separate tasks.

### Architectural Context
- Semantic validation belongs in the service, not the handler. Handler does null/shape; service owns format and bounds.
- `UserService.ValidateEmail` is the canonical example of this split — match its shape.

### Relevant Files
- `internal/user/service.go` — add `ValidatePhone` here.
- `internal/user/service_test.go` — add tests here.
- `internal/pkg/validate/phone.go` — read-only reference for `PhoneE164` and `validate.Error`.

### Reference Implementation
Mirror `UserService.ValidateEmail` in `service.go`:
- Signature: `func (s *UserService) ValidatePhone(phone *string) error`.
- Nil pointer → return nil. Empty string → return error.
- Return the `*validate.Error` from `PhoneE164` unwrapped — no `fmt.Errorf`.
- Copy the table-driven layout from `TestUserService_ValidateEmail`.

### Constraints
- Use `validate.PhoneE164`. No regex, no new dependencies.
- Don't touch `UserRepository` or its mock — validation is pure.
- Don't wrap the error; the handler relies on `errors.As(&validate.Error{})` to map it to HTTP 422.

### Non-Goals
No handler, migration, sqlc, or integration-test changes. No edits to `ValidateEmail` or other unrelated methods.

### Edge Cases
- `phone == nil` → return nil (field not provided).
- `*phone == ""` → return `validate.Error` (malformed input).
- Strict E.164: `1234567890` (no leading `+`) must fail.
- The handler already checks the JSON field is present and is a string — don't re-check those concerns here.

### Acceptance Criteria
1. `ValidatePhone(phone *string) error` on `UserService`.
2. `nil` phone → returns nil.
3. Empty or non-E.164 → returns `*validate.Error` (verifiable via `errors.As`).
4. Valid E.164 (e.g., `+14155552671`) → returns nil.
5. At least four test cases: valid, invalid, nil, empty.
6. Only `service.go` and `service_test.go` change.

### Verification
    go test ./internal/user/... -v -run TestValidatePhone
    go vet ./...
    golangci-lint run ./internal/user/...

Why This Works

References

1. Anthropic — Effective Context Engineering for AI Agents¹ — Why just-in-time context retrieval and focused instructions outperform pre-loading everything into the prompt.
2. Claude Code Docs — Best Practices⁸ — Including verification commands and CLAUDE.md conventions so the agent can self-check its work.
3. Claude Directory — Context Engineering for Claude Code³ — The task trifecta: state the goal, provide constraints, define done.
4. Augment Code — Best Practices for Using AI Coding Agents⁶ — Pointing agents at reference implementations and reviewing changes after each sub-task.
5. JetBrains — Coding Guidelines for Your AI Agents⁵ — How missing constraints lead agents to skip pagination, misuse injection patterns, and ignore project conventions.
6. Google Cloud — Five Best Practices for AI Coding Assistants⁷ — Planning-first workflow and using tests as acceptance criteria for generated code.
7. HumanLayer — Writing a Good CLAUDE.md² — Why fewer, focused instructions outperform instruction overload, and the ~150–200 instruction ceiling for frontier models.
8. Chroma — Context Rot (Hong et al., 2025)⁴ — Empirical study across 18 LLMs showing that attention to context degrades non-uniformly as input length grows.