Anthropic's Designs Three-Agent Harness Supports Long-Running Full-Stack AI Development
Leela Kumili · rss · 2026-04-06 source ↗ Reader 🖥️
Agent HarnessContext Rot & Context LimitsLong-Running & Managed AgentsMulti-Agent OrchestrationRiding Model Improvements
Splitting long-running AI development tasks among three specialized agents — planner, generator, and evaluator — with structured handoff artifacts solves the two core failure modes of autonomous coding sessions (context loss and self-evaluation bias), producing higher-quality and more consistent outputs.
practical · expository — A journalistic summary of an Anthropic engineering blog post. Describes an engineering pattern and its rationale. Practical: argues that practitioners should adopt this harness design. Notably thin on original analysis — the real text is the Anthropic engineering post, not this summary.
Structure
| Part | Role |
|---|---|
| Problem framing: context loss and premature termination | Establishes the pain point that motivates the harness design |
| Context reset mechanism vs. compaction | Technical distinction — justifies why resets with handoff artifacts are preferred over preserving context |
| Evaluator agent and self-evaluation bias | Core insight — explains why a separate evaluator is necessary and how it is calibrated |
| Frontend design application (grading criteria, Playwright MCP, iterative cycles) | Case study / evidence that the design works on subjective tasks |
| Operational notes: human oversight, parallel/sequential execution | Pragmatic caveats — limits the scope of automation claims |
| Future outlook: models improve, harness evolves | Forward-looking context — harness design is not static |
Key terms
- harness Agent Harness
A structured multi-agent workflow that coordinates distinct agents (planner, generator, evaluator) through defined handoffs — the scaffolding around the AI, not the AI itself - context reset Context Rot & Context Limits
Deliberately clearing the agent's context at a defined boundary and handing over a structured artifact (state summary, progress record) so the next agent starts fresh from a known state — as opposed to compaction, which attempts to preserve continuity - evaluator agent Multi-Agent Orchestration
A separate AI agent whose sole job is to judge the generator's output using predefined criteria and few-shot examples; it navigates live pages and provides detailed critiques rather than passively reviewing text - compaction Context Rot & Context Limits
A technique for preserving context by summarizing or compressing prior conversation, allowing the model to appear to have longer memory — contrasted with context resets because it causes models to grow cautious as limits approach
Propositions
- explicitestablishedAI agents consistently overrate their own outputs, especially on subjective tasks like designconf: highsupports ←How to turn your AI into a world-class designer · p5 — Concrete design-domain confirmation of the general claim 'AI agents consistently overrate their own outputs, especially on subjective tasks like design' — this piece's Technique 3 is built entirely around that failure mode.supports ←Podcast: Strands Agents with Clare Liguori · p5 — Gives the practitioner rationale (same-model self-agreement bias) behind the empirical finding that agents overrate their own outputs, and prescribes a fix: judge with a different model.
- explicitcorroboratedSeparating the agent doing the work from the agent judging it is a strong lever for improving output qualityconf: highextends ←How to turn your AI into a world-class designer · p6 — Worked, design-specific application of the general principle 'separating the agent doing the work from the agent judging it is a strong lever for improving output quality' — adds a fresh-context-only, screenshot-scoped critic protocol with explicit stopping criteria.supports ←Presentation: The Multi-Agent Approach: Building Reliable and Controllable Software Development Automation · p14 — Same lever: separating the agent doing the work from the agent judging it improves output quality.
- explicitdevelopedContext loss is the primary cause of failure in long-running AI agent sessionsconf: highextends ←Yonathan Cohen's Post · p1 — Gives a concrete, independently-built example of the general claim that structured handoff artifacts enable coherent continuation after a context/session reset.
- explicitdevelopedStructured handoff artifacts allow coherent continuation after context resets without the caution-inducing effects of compactionconf: highextends ←The Production Gap: 5 Patterns for Building Long-Running AI Agents · p3 — Generalizes the structured-handoff-artifact idea into a full data-pipeline discipline: checkpointing, partial-failure handling, and idempotency for long-running agents.
- explicitHuman oversight remains necessary for initial calibration and quality validation even in automated harnessesconf: high
- explicitAs AI models improve, the harness's role will shift — some tasks will move to next-generation models directly, while improved models will enable harnesses to tackle more complex workconf: medium
- implicitcorroboratedGood harness design requires ongoing experimentation and adjustment as model capabilities evolve — there is no stable final configurationconf: highsupports ←Yonathan Cohen's Post · p1 — The five-item checklist (decisions, verbatim draft, rejected paths, tone rules, next instruction) is a concrete answer to 'what state information is captured' — the design variable that source names as critical.
- implicitThe quality of the handoff artifact — what state information is captured between agents — is the critical design variable in a context-reset harnessconf: medium
Arguments
A separate evaluator agent is necessary for quality control in long-running AI development strength: medium
Explicit premises:
- AI agents overrate their own outputs, especially on subjective tasks
- Self-evaluation fails on design tasks because agents cannot reliably judge their own work
Implicit premises:
- Separation of concerns (producer vs. assessor) is a reliable quality principle borrowed from human engineering practice
- Calibration through few-shot examples is sufficient to make an evaluator agent reliable — its own potential biases are not addressed
Inferences:
- The evaluator pattern could apply to any AI task where output quality is hard to specify in advance — not just frontend design
- The article does not address whether the evaluator itself needs evaluation — a regress it assumes away
Context resets with structured handoff artifacts outperform compaction for long-running sessions strength: low
Explicit premises:
- Compaction makes models cautious as they approach context limits, potentially affecting performance
- Context loss causes task failure or incoherence in long sessions
Implicit premises:
- A well-designed handoff artifact captures sufficient state to resume the task — this assumption bears all the weight but is not examined
- The caution induced by compaction is a meaningful performance degradation, not a minor artifact
Inferences:
- The design quality of the handoff artifact is the real bottleneck — the article doesn't address what must be in it
- Context reset + handoff is itself a form of state management, implying the agent system needs a formal state model for the task being performed
Narratives
- Anthropic's frontend design pipeline: evaluator agent navigates live pages using Playwright MCP, interacts with the interface, scores on four criteria (design quality, originality, craft, functionality), and provides critiques that drive 5-15 iteration cycles per run, sometimes taking up to four hours
→ Demonstrates that the evaluator agent pattern produces 'designs that combine visual distinction with functional accuracy' — the primary evidence claim for the three-agent design - Artem Bredikhin's LinkedIn observation: 'every new context window is amnesia. The breakthrough is structure: JSON feature specs, enforced testing, commit-by-commit progress, and an init script that ensures every session starts with a working app'
→ External validation of the article's central claim — structured artifacts solve the amnesia problem. Also reveals what a handoff artifact might concretely contain (JSON specs, tests, git state, init script)
Questions addressed
- ✅ How can AI agents maintain coherence and make progress over multi-hour sessions without context degradation?
Context resets with structured handoff artifacts allow each session to start from a defined, known state - ✅ How can AI agents produce reliable evaluations of their own output on subjective tasks?
Separate evaluator agent calibrated with few-shot examples and scoring criteria - ⭕ What specifically must a handoff artifact contain to enable coherent session continuation?
- ⭕ How should evaluation criteria be derived and validated for a given task type?
Gaps
- incompleteThe article is a summary of an Anthropic engineering blog post and contains almost no original analysis. The critical engineering details — what handoff artifacts must contain, how grading criteria are derived, how the evaluator is calibrated — are referenced but not explained. This is a pointer to the source post, not a standalone analysis.
Read next: Read the original Anthropic engineering blog post directly: anthropic.com/engineering/harness-design-long-running-appsclosed byYonathan Cohen's Post — Offers a concrete, if anecdotal and third-party (not Anthropic's own), specification of what a handoff artifact must contain — partially answering the gap's 'what handoff artifacts must contain' question, though without the grading/evaluator detail the gap also names. - incompleteThe article doesn't address what happens when the evaluator agent is wrong. If the evaluator has calibration errors, the generator will iterate toward those errors. The regress (who evaluates the evaluator?) is assumed away.
Read next: Literature on specification gaming in RL; AI alignment work on reward hacking - uninformedNo comparison to alternative approaches. The context-reset claim is made without benchmarks against compaction or other memory strategies. The practical superiority is asserted, not demonstrated.
Read next: Anthropic's research on long-context models; MemGPT and other memory architecture papers - incompleteThe 'harness combinations evolve with model capabilities' closing observation implies the current design is temporary, but there's no framework for deciding when to change the harness or what signals indicate that a capability shift has occurred.
Read next: AI systems design literature on graceful degradation and capability thresholdsclosed byHow Anthropic's product team moves faster than anyone else | Cat Wu (Head of Product, Claude Code) — Supplies the missing when-to-change-the-harness signal: at each new model generation Anthropic removes prompting interventions and crutches no longer needed, using model introspection on its own mistakes to find what to fix (p5, p10).
Related documents
- Harness design for long-running application development
- How to Actually Prompt Fable 5, Straight From the Team That Built It
- Prompting Claude Fable 5
- Agent Harness Engineering
- Accelerating Delivery: How AI Agents Can Own the Initial Code Draft
- The Anatomy of an Agent Harness
- A harness for every task: dynamic workflows in Claude Code
- Don't train the model, evolve the harness
- Google's Genkit Ships Agents API with Detached Turns and Human-in-the-Loop for TypeScript and Go
- No Vibes Allowed: Solving Hard Problems in Complex Codebases – Dex Horthy, HumanLayer