The Anatomy of an Agent Harness

Akshay · tweet · 2026-07-08 Reader 🖥️

practical · expository — Expository/taxonomic: it defines a term (the harness), enumerates ~12 components and 7 design-decision axes, traces one execution cycle, and surveys five frameworks. It largely ASSERTS a synthesized consensus rather than argues; the two genuine mini-arguments (separate permission from reasoning; name the parts so you can debug/evolve them) are light. Prescriptive intent throughout ('the next time your agent fails, look at the harness'), so practical.

Structure

Key terms

• agent harness Agent Harness
  The complete non-model software infrastructure wrapping an LLM — orchestration loop, tools, memory, context management, state persistence, error handling, guardrails, verification. 'If you're not the model, you're the harness.' The harness is the machinery; the 'agent' is the emergent goal-directed behavior it produces. Von Neumann framing: the harness is the operating system around the model-as-CPU.
• orchestration loop (TAO / ReAct / 'dumb loop')
  The heartbeat: assemble prompt, call LLM, parse output, execute tool calls, feed results back, repeat until done. Mechanically 'often just a while loop'; Anthropic's is a 'dumb loop' where all intelligence lives in the model — 'the complexity lives in everything the loop manages, not the loop itself.'
• context management & context rot
  The component that keeps the working set small and high-signal; 'where many agents fail silently.' Context rot = performance degrading 30%+ when key content lands mid-window (Chroma / 'Lost in the Middle'), even in million-token windows. Countermeasures: compaction, observation masking, just-in-time retrieval, subagent delegation; goal is the 'smallest possible set of high-signal tokens.'
• memory (multi-timescale) Agent Memory
  State kept across turns and sessions at multiple timescales: short-term (session history), long-term (CLAUDE.md/MEMORY.md, LangGraph Stores, OpenAI Sessions). Claude Code's three-tier hierarchy (lightweight always-loaded index -> on-demand topic files -> search-only raw transcripts) with the design principle that the agent treats its own memory as a 'hint' and verifies against actual state before acting.
• verification loops
  The component that 'separates toy demos from production agents': rules-based feedback (tests, linters, type checkers), visual feedback (screenshots via Playwright), and LLM-as-judge (a separate subagent). Cherny: giving the model a way to verify its work improves quality 2-3x. Framed (via Fowler/Thoughtworks) as guides (feedforward) vs sensors (feedback).
• guardrails & the permission/reasoning separation
  Safety enforced architecturally separate from model reasoning: the model decides what to ATTEMPT, the tool system decides what is ALLOWED. Claude Code gates ~40 capabilities across trust/permission/confirmation stages; OpenAI runs input, output, and per-tool guardrails with a 'tripwire' that halts the agent.
• subagent orchestration
  Spawning bounded sub-runs: Claude Code's Fork (byte-identical context copy), Teammate (separate pane, file-mailbox comms), Worktree (own git branch); OpenAI's agents-as-tools and handoffs; LangGraph's nested state graphs. Each subagent explores extensively but returns a condensed 1-2k-token summary (a context-management move as much as a topology one).
• harness thickness / scaffolding
  How much logic lives in the harness vs the model. Scaffolding is TEMPORARY construction infrastructure, removed when the building is complete; as models improve, harness complexity should shrink (Anthropic deletes planning steps as models internalize them; Manus rebuilt 5x). The 'future-proofing test': a design is sound if performance scales with stronger models WITHOUT added complexity.
• three levels of engineering (prompt ⊂ context ⊂ harness)
  Concentric scopes: prompt engineering crafts the instructions; context engineering manages what the model sees and when; harness engineering encompasses both plus tool orchestration, state, error recovery, verification, safety, lifecycle. 'The harness is not a wrapper around a prompt.'
• the seven design decisions
  The axes every harness architect chooses along: single- vs multi-agent, ReAct vs plan-and-execute, context-window strategy, verification-loop design, permission/safety architecture, tool-scoping, harness thickness. A decision space layered on top of (and partly restating) the component list.

Propositions

• explicitdevelopedThe agent harness is the complete non-model software wrapping an LLM (orchestration loop, tools, memory, context management, state, error handling, guardrails, verification); the 'agent' is the emergent behavior, the harness is the machinery — 'if you're not the model, you're the harness,' and 'when someone says I built an agent, they mean they built a harness and pointed it at a model.'conf: high
  refines ←Podcast: Strands Agents with Clare Liguori · p10 — Sharpens the broad 'if you're not the model, you're the harness' split into a three-way distinction -- model, thin framework loop, and harness -- defining 'harness' specifically as what enables long-running/complex work, not just 'everything non-model.'
• explicitEngineering the model's environment nests in three concentric levels: prompt engineering (the instructions) is inside context engineering (what the model sees and when), which is inside harness engineering (both plus tool orchestration, state, error recovery, verification, safety, lifecycle). The harness is not a wrapper around a prompt.conf: high
• explicitcorroboratedA production harness decomposes into a fixed set of named components — the prose walks through eleven (orchestration loop, tools, memory, context management, prompt construction, output parsing, state management, error handling, guardrails/safety, verification loops, subagent orchestration) under a 'twelve components' heading — synthesized across Anthropic, OpenAI, LangChain and the practitioner community.conf: high
  supports ←The Anatomy of an Agent Harness · p1 — Both this article and the corpus's existing Akshay-tweet synthesis independently arrive at a decomposed, named-component view of the harness, confirming the decomposition is a convergent practitioner framing rather than one author's idiosyncratic model — though this article derives each component from a specific model limitation rather than presenting a static checklist.
• explicitThe orchestration loop is mechanically trivial — 'often just a while loop,' a 'dumb loop' where all intelligence lives in the model; the harness's complexity lives in everything the loop MANAGES (tools, context, memory, verification), not in the loop itself.conf: high
• explicitestablishedContext management is where agents fail silently: context rot degrades performance 30%+ when key content sits in mid-window positions (Chroma; Stanford 'Lost in the Middle'), even in million-token windows; the fix is not a bigger window but keeping the smallest set of high-signal tokens via compaction, observation masking, just-in-time retrieval, and subagent delegation.conf: high
  supports ←No Vibes Allowed: Solving Hard Problems in Complex Codebases – Dex Horthy, HumanLayer · p5 — Corroborates, from a practitioner framing, the measured finding that context degrades performance substantially once key content sits mid-window ('lost in the middle').
  supports ←Stop Using /init for AGENTS.md · p3 — Both cite the same context-degradation research lineage (Lost in the Middle / Levy et al.) to argue that adding content, even accurate content, has a real attention cost distinct from whether the content is 'wrong.'
  extends ←The Anatomy of an Agent Harness · p5 — The Akshay synthesis cites '30%+ context rot' as an empirical headline figure; this article gives the mechanism and three specific named mitigations (compaction, tool-call offloading, skills/progressive-disclosure) that operationalize the same context-rot problem.
• explicitErrors compound multiplicatively — a 10-step process at 99% per-step success still yields only ~90.4% end-to-end — so the harness must classify failures (transient->retry-with-backoff, LLM-recoverable->return-as-ToolMessage, user-fixable->interrupt, unexpected->bubble-up) and cap retries (Stripe caps at two).conf: high
• explicitcorroboratedVerification is what separates toy demos from production agents: rules-based feedback (tests/linters/type checkers), visual feedback (screenshots), and LLM-as-judge (a separate subagent); giving the model a way to verify its own work improves quality 2-3x (Cherny), and verification is best framed as guides (feedforward) versus sensors (feedback).conf: high
  supportsHarness engineering for coding agent users · p3 — Strongest cross-author link — the anatomy explicitly cites 'Martin Fowler's Thoughtworks team' framing verification as guides (feedforward) versus sensors (feedback). Boeckeler p3: feedforward-only or feedback-only harnesses are each insufficient; both control directions are necessary. Same source, same claim, arriving from Akshay's synthesis.
  supports ←Presentation: From AI Agent Demo to Production: Automated Testing and Evaluation · p7 — Both name automated, CI-integrated verification (rules-based checks plus LLM-as-judge) as what actually separates toy demos from production-ready agents.
  extends ←How we Claude Code · p13 — Extends the harness piece's rules-based/visual/LLM-as-judge verification taxonomy with a concrete implementation pattern: publishing component state directly into the DOM as a schema+fixtures+invariants contract, so rule-based checks, human visual review, and agent-driven checks all read the same underlying artifact.
  extends ←Presentation: The Agent Harness: Control Planes, Invariants, and Approval Boundaries for Production AI Agents · p12 — Sharpens 'verification separates toy demos from production agents' into a stronger, more specific claim: verification is not enough unless proof specifically crosses the user-visible edge, not merely an internal rules-based/LLM-as-judge check.
• explicitestablishedPermission and safety should be enforced architecturally SEPARATE from model reasoning: the model decides what to attempt, the tool system decides what is allowed (Claude Code gates ~40 capabilities across trust/permission/confirmation stages; OpenAI runs input/output/tool guardrails with a halting 'tripwire').conf: high
  supports ←Podcast: Strands Agents with Clare Liguori · p4 — Steering hooks plus the ledger are a concrete implementation of an architecturally-separate enforcement layer, adding a stateful cross-call mechanism this component list names but doesn't specify.
  supports ←Zero to Agent in 30 Minutes: Build a Sports Concierge Agent with Chester Ismay · p4 — Chester's permission-scoping is a lightweight, concrete instance of the architectural principle that permission/safety enforcement should sit outside the model's own reasoning.
• explicitcorroboratedEvery production framework implements the same underlying pattern — an LLM-call node and a tool node joined by a conditional edge that loops until no tool calls remain — so the harness is a convergent, real architecture rather than an ad-hoc wrapper (Anthropic query()/dumb loop, OpenAI Runner + Codex three-layer, LangGraph state graph, CrewAI roles, AutoGen conversation patterns).conf: high
  supports ←Google's Genkit Ships Agents API with Detached Turns and Human-in-the-Loop for TypeScript and Go · p1 — Genkit's single chat() object handling one-shot, streamed, paused, and detached modes is another instance of the corpus's observation that production frameworks converge on the same underlying loop/state-graph pattern rather than inventing bespoke architectures.
• explicitdevelopedBeyond the components, every harness architect faces seven design decisions, each a spectrum with empirical tradeoffs: single- vs multi-agent (maximize single first; split past ~10 overlapping tools), ReAct vs plan-and-execute (LLMCompiler 3.6x speedup), context-window strategy, verification design, permission architecture, tool scoping (Vercel removed 80% of v0's tools and improved; ACON 26-54% token reduction at 95%+ accuracy), and harness thickness.conf: high
  supportsAgent Harness Engineering · p8 — Osmani p8: 'ten focused tools outperform fifty overlapping ones because the model can hold the menu in its head.' Anatomy decision 6 (tool scoping) supplies concrete evidence: Vercel removed 80% of v0's tools and got better results; Claude Code achieves 95% context reduction via lazy loading; 'expose the minimum tool set needed for the current step.'
  refines20 Loop Design Patterns Every AI Engineer Should Know · p11 — 20-loops p11 (Plan -> Execute -> Replan) is listed with no cost or tradeoff (that thread's g2 = no failure modes/costs per loop). Anatomy decision 2 (ReAct vs plan-and-execute) refines it by supplying the tradeoff and a number: ReAct interleaves reasoning and action every step (flexible, higher per-step cost); plan-and-execute separates planning from execution; LLMCompiler reports a 3.6x speedup over sequential ReAct.
  extends ←The Self-Building Agent: A LangChain4j Experiment · p3 — Adds a second, independent empirical data point (LangChain4j supervisor-vs-workflow, ~3x) for the same general effect the harness-anatomy piece cites via LLMCompiler's ~3.6x ReAct-vs-plan-and-execute speedup: removing dynamic, LLM-decided step-by-step coordination in favor of a pre-planned/declared execution path yields a large, similarly-sized speedup.
• explicitdevelopedHarness complexity should DECREASE as models improve — scaffolding is temporary infrastructure removed when the building is complete (Manus rebuilt 5x, each rewrite removing complexity); the 'future-proofing test' is that a design is sound if performance scales up with more powerful models WITHOUT added harness complexity.conf: high
  supportsHarness design for long-running application development · p7 — Anthropic p7: when a stronger model lands, remove harness components one at a time, measure impact, and keep only the load-bearing pieces. Anatomy generalizes the same prescription via the scaffolding metaphor and future-proofing test (complexity should shrink as models improve; Manus rebuilt 5x removing complexity).
  supportsPrompting Claude Fable 5 · p2 — Fable-5 p2: treat a capability jump as a prompt to re-evaluate and PRUNE existing instructions, tools, and guardrails. Anatomy supplies the principled basis: scaffolding is temporary, removed when the building is complete, and Anthropic deletes planning steps from Claude Code's harness as new model versions internalize that capability.
  extends ←Stop Overengineering Your Agent Harness · p6 — Extends the 'harness complexity should decrease as models improve' claim with a named mechanism (the Kirby effect: each component encodes an expiring assumption) plus additional concrete precedents (CoT->reasoning models, Amp's plan-mode removal).
• explicitModel and harness co-evolve: models are increasingly post-trained with specific harnesses in the loop, so the coupling is tight — Claude Code's model learned to use its specific harness, and changing tool implementations can degrade performance.conf: high
  supportsAgent Harness Engineering · p5 — Osmani p5: 'models get post-training coupled to the harness they were trained against, creating overfitting to specific tool implementations.' Anatomy's co-evolution principle states exactly this — Claude Code's model learned its specific harness, and changing tool implementations can degrade performance.
• explicitThe harness is the product: two products on identical models can differ wildly in performance based solely on harness design (TerminalBench: a harness-only change moved agents 20+ ranking positions), so the harness is where the hard engineering lives, not a commodity layer — even as the field trends toward thinner harnesses, the harness never disappears.conf: high
  supportsAgent Harness Engineering · p1 — Osmani p1: 'a decent model with a great harness beats a great model with a bad harness.' This piece's 'the harness is the product' with the TerminalBench 20-rank jump from a harness-only change (identical weights) is the same claim with a hard benchmark number.
• implicitThe taxonomy's whole utility rests on an unstated separability premise — that these components are distinct, individually nameable, and independently improvable — which is precisely the condition that lets the sibling strategy's automated loop 'add exactly one mechanism per iteration': the anatomy supplies the labeled search space (the WHAT) that the fitness-selection strategy searches over (the HOW).conf: high
  supportsDon't train the model, evolve the harness · p4 — CENTERPIECE. The sibling's strategy automates harness improvement with a proposer that 'adds exactly one mechanism per iteration' and an outer loop that keeps it only if it beats the current best. This anatomy supplies WHAT those mechanisms are: the named components and seven decision axes are the labeled search space the fitness-selection loop searches over. Anatomy = the what; strategy = the search. The sibling's note says this deep-dive was written EARLIER, so it is the prior/foundational piece the strategy stands on — hence 'supports' (not refines/extends, which would imply it postdates and improves the strategy). The load-bearing hinge is p14's separability premise: one-mechanism-at-a-time selection is only valid if the components are actually decoupled (see g1/g3).

Arguments

Narratives

• Beren Millidge's 2023 'Scaffolded LLMs as Natural Language Computers': a raw LLM is a CPU with no RAM, disk, or I/O; the context window is RAM (fast, limited), external databases are disk (large, slow), tool integrations are device drivers, and the harness is the operating system — 'we have reinvented the von Neumann architecture.'
  → The conceptual anchor — makes the harness's role precise by mapping each component onto a known computing-architecture part, and justifies why a harness is a natural, not incidental, abstraction.
• LangChain changed only the infrastructure wrapping their LLM (same model, same weights) and jumped from outside the top 30 to rank 5 on TerminalBench 2.0; a separate research project reached 76.4% pass rate by having an LLM optimize the infrastructure itself, surpassing hand-designed systems.
  → The empirical hook that the harness — not the model — drives benchmark rank; motivates naming and engineering the harness. (Also relayed in the sibling as corroboration.)
• Claude Code's memory is a three-tier hierarchy — a lightweight ~150-char-per-entry index always loaded, detailed topic files pulled in on demand, and raw transcripts reachable only via search — and a critical principle: the agent treats its own memory as a 'hint' and verifies against actual state before acting.
  → A component done well: shows memory as a progressive-disclosure structure plus the anti-hallucination discipline (verify the hint), i.e. memory and context management working together.
• Manus was rebuilt five times in six months, each rewrite removing complexity: complex tool definitions became general shell execution; 'management agents' became simple structured handoffs.
  → Demonstrates that evolving a harness includes SHRINKING it as models absorb capability — the concrete basis for the scaffolding-removal / future-proofing stance.

Questions addressed

• ✅ Concretely, what is an agent harness — what is it made of?
  The complete non-model infrastructure, decomposed into ~12 named components and framed via the von Neumann analogy (harness = OS) and the three concentric engineering levels (prompt ⊂ context ⊂ harness).
• ✅ Why do agents that work in demos fall apart in production?
  The failures live in the harness, not the model — forgetting (memory), silent tool failures (error handling), context filling with garbage (context management), compounding step errors — each mapped to a specific component you can name and fix.
• ✅ How do the components actually work together in a single cycle?
  The seven-step walkthrough — prompt assembly, LLM inference, output classification, tool execution (validate/permission/sandbox/capture), result packaging, context update (triggering compaction near limits), loop — with layered termination conditions and a Ralph Loop for multi-window tasks.
• ✅ Should you build a thick harness or a thin one, and how should it change as models improve?
  Trend thin and shrink over time: scaffolding is temporary, removed as models internalize capabilities; pass the future-proofing test (performance scales with stronger models without added complexity).

Gaps

• incompleteThe decomposition is not orthogonal and specifies no module boundaries. Four of the named components — memory (3), context management (4), prompt construction (5), and state management (7) — are views of a single 'what the model sees / what persists' concern (Claude Code's git-commits-as-checkpoints sits under state, MEMORY.md under memory, compaction under context, prompt assembly under construction, but they are one persistence-and-window problem wearing four hats). Guardrails/permission (9) is not disjoint from tools (2) — the walkthrough runs the permission check INSIDE tool execution (step 4). Output parsing (6) is near-vestigial under native tool calling (a sub-step of the loop, not a peer module). And the orchestration loop (1) is a LEVEL above the contents it manages, not a coequal component ('the complexity lives in everything the loop manages'). The prose enumerates eleven items under a 'twelve components' heading. So the list reads as a practitioner checklist organized by mechanism and vendor vocabulary, not a coupling-minimizing partition into high-cohesion modules — it cannot tell you where to actually cut the system.
  Read next: Read Parnas ('On the Criteria To Be Used in Decomposing Systems into Modules'), Yourdon & Constantine on coupling/cohesion, and Ousterhout on deep modules; re-derive the harness as narrow-interface deep modules and test which of the eleven are genuinely separable versus one persistence/context responsibility split four ways — the structured-design question the piece's naming step presupposes but never answers.
• incompleteAnatomy without failure-mode interactions. The seven-step walkthrough gives happy-path dynamics only; the piece never addresses how components interfere when things go wrong — compaction (context management) discarding state the verifier still needed, a guardrail tripwire firing mid-tool-call and leaving state half-written, or a memory 'hint' that contradicts actual state after a context reset. The hardest engineering lives in these cross-component failure interactions, which the static list and the clean walkthrough elide.
  Read next: Read extract-anthropic-harness-design-long-running-application-development and extract-anthropics-three-agent-harness on context-reset handoff artifacts and evaluator calibration; look for how components must expose contracts to survive each other's failures rather than only cooperating on the happy path.
• incompleteNo interface or contract between components is specified. The piece names components but never states what each EXPOSES to the others — what memory hands to context management, what guarantees the loop makes to the tool layer, what verification consumes and returns. Without interface contracts, the 'independently improvable' premise (p13/p14) that the sibling's fitness-selection strategy depends on is unverified: 'add one mechanism' only stays local if the components are actually decoupled, which the piece asserts by layout rather than by design.
  Read next: Read on module interfaces/APIs (information hiding, contracts) and re-examine the sibling's automated 'add one mechanism, keep if better' loop to check whether a change to one component ripples across the coupled memory/context/state cluster.
• incompleteTwo overlapping taxonomies whose relationship is left unstated. The twelve components and the seven decisions substantially overlap: decisions 3/4/5/6 (context-window strategy, verification design, permission architecture, tool scoping) simply re-open components 4/10/9/2 as 'choices,' while only decisions 1/2/7 (single-vs-multi-agent, ReAct-vs-plan-execute, harness thickness) are architectural choices not already a component. The piece presents both frameworks without saying whether the seven decisions are just the tunable parameters of a subset of the twelve components.
  Read next: Consolidate into one model — components as nouns (what the harness contains) versus configuration axes as verbs (how each is set) — and test whether 'seven decisions' collapses into 'the tunable dials on four of the components,' leaving only single-vs-multi-agent, ReAct-vs-plan, and thickness as genuinely first-order architecture.
• incompleteSecond-hand synthesis with no first-party build and no eval of the taxonomy itself. Every component and framework claim is assembled from vendor docs, and the empirical figures (30%+ context rot, 2-3x verification, 3.6x LLMCompiler, TerminalBench 20-rank jump, 76.4%, 80% tool removal, 26-54% token reduction) are all relayed. There is no worked minimal harness, no ablation showing these components are necessary or sufficient, and no test of the '12' as a complete or minimal set.
  Read next: Build a minimal harness against the eleven components and ablate each to test necessity/sufficiency; go to the primary sources (Chroma context-rot study, the LangChain TerminalBench writeup, the LLMCompiler and ACON papers) before treating the relayed figures as load-bearing.
  closed byPodcast: Strands Agents with Clare Liguori — Supplies a genuine first-party ablation study -- Clare's own case study comparing workflow, simple instructions, and steering hooks across hundreds of evaluations, with real pass-rate numbers (82.5%, 100%) -- partially answering the gap's call to 'build a minimal harness and ablate' rather than relay vendor-reported figures. Partial: it ablates control-flow techniques, not the eleven named components specifically, and the workflow approach's own numeric score is still not reported.

Closes gaps elsewhere

• Agent Harness Engineering · g1 — PARTIAL. Osmani g1 asks when harness complexity itself becomes a liability — the scaling problem of accumulating ratchet rules, and at what point the constraint set is too large. The anatomy supplies a directional criterion: the 'future-proofing test' (a design is sound only if performance scales up with more powerful models WITHOUT added harness complexity) plus the scaffolding-shrinks stance (Manus rebuilt 5x removing complexity; Anthropic deletes planning steps as models internalize them). So complexity that a stronger model would render redundant is, by this test, a liability. It does NOT answer the quantitative half of g1 — no threshold for 'too large for the model to follow / the engineer to maintain,' no maintenance-cost model — so this is a design principle, not the boundary the gap asks for.

Related documents

• 11 Tips For AI Coding With Ralph Wiggum
  Agent Harness Agentic Loop Context Rot & Context Limits Feedback Loops
• 20 Loop Design Patterns Every AI Engineer Should Know
  Agent Memory Agentic Loop Evals Feedback Loops
• Agent Harness Engineering
  Agent Harness Agent Tool Management Agentic Loop Context Rot & Context Limits
• Don't train the model, evolve the harness
  Agent Harness Agentic Loop Context Engineering Evals
• Stop Overengineering Your Agent Harness
  Agent Harness Agent Tool Management Context Engineering Guardrails
• Presentation: From AI Agent Demo to Production: Automated Testing and Evaluation
  Evals Feedback Loops Guardrails Multi-Agent Orchestration
• Accelerating Delivery: How AI Agents Can Own the Initial Code Draft
  Agent Harness Context Rot & Context Limits Guardrails Multi-Agent Orchestration
• [FULL WORKSHOP] AI Coding For Real Engineers - Matt Pocock, AI Hero
  Agentic Loop Context Rot & Context Limits Feedback Loops Multi-Agent Orchestration
• AI Root Cause Analysis Shifts from Model Reasoning to Context Engineering
  Agent Harness Agentic Loop Context Engineering Evals
• Harness design for long-running application development
  Agent Harness Context Rot & Context Limits Evals Multi-Agent Orchestration