The Work Behind Delegation: A Framework for Supervising AI Coding Agents

Yeon Su Park; Nadia Arvi; Hae Ri Lee; Sehoon Lim; Qianou Ma; Juho Kim · pdf · 2026-09-22 Reader 🖥️

Agent Planning Artifacts Context Store Hooks Human-in-the-Loop Multi-Agent Orchestration Scope Discipline Supervisory Programming

Developer supervision of AI coding agents is not a single activity performed at fixed checkpoints but a connected process organized into seven distinct, loop-connected stages (Plan, Monitor, Wait, Review, Teach, Manual Fix, Update Assets) -- a reconfiguration of Sheridan's classical five-stage human-supervisory-control framework made necessary by agents' greater autonomy -- and developers manage the resulting cross-stage demands (alignment, quality, understanding) by concentrating effort in planning, distributing supervisory work to other agents and automated hooks, and externalizing recurring corrections into reusable assets.

theoretical · mixed — An HCI research paper: primarily theoretical/descriptive (building and empirically validating a framework for how developer supervision of AI coding agents is organized), with a practical coda (design implications for agent builders and tool builders) drawn from the theoretical findings rather than argued independently.

Structure

Part	Role
Introduction, related work, and Sheridan's background framework	Motivation and theoretical starting point
Study 1: qualitative study with 19 developers (think-aloud coding session + workflow-diagramming activity)	Evidence -- grounds the seven-stage framework in direct observation
The seven-stage framework itself (Plan, Monitor, Wait, Review, Teach, Manual Fix, Update Assets) and its loops	Core theoretical contribution
Study 2: applying the framework to 102 Reddit threads (12,912 comments) across three developer subreddits	Evidence at scale -- tests whether the framework's demands/strategies generalize beyond the lab study
Findings: three cross-stage demands (alignment, quality, understanding) and three management strategies (concentrate in Plan, distribute, externalize into assets)	Core empirical findings
Discussion: applicability beyond software, design implications, further development; Limitations	Extension and qualification of the findings

Key terms

supervisory stage
One of seven distinct phases of developer activity when overseeing a delegated agent task (Plan, Monitor, Wait, Review, Teach, Manual Fix, Update Assets), each with its own supervisory goal, distinguished empirically from Sheridan's five by how developers themselves segmented and diagrammed their own workflows.
Wait
A newly identified stage, distinct from Monitor, in which a developer deliberately disengages attention from a running agent -- turning to another task, another agent, or an unrelated activity -- because they trust the agent to continue productively without observation for an extended period.
Review
A distinct stage (absent as a separate category in Sheridan's framework, where assessment is folded into Monitor) in which developers assess a completed, bounded chunk of agent work against their original intent -- checking for the gap between intent and result that, per participants, 'only becomes visible once you get to this stage.'
Update Assets
An optional stage, replacing Sheridan's Learn, in which developers externalize lessons from the current task into persistent, shared resources (AGENT.md/CLAUDE.md files, documentation, linters, retrospective-generated skill files) that feed forward into future Plan stages -- learning encoded into the system rather than retained only in the supervisor's own memory.
delegated supervision
Supervisory loops (monitoring, review, correction) configured in advance to run via other agents or deterministic hooks without developer involvement at each occurrence -- distinguished from collaborative supervision (shared with an agent) and developer-retained supervision (kept entirely by the human).

Propositions

explicitDeveloper supervision of AI coding agents is better modeled as seven connected stages (Plan, Monitor, Wait, Review, Teach, Manual Fix, Update Assets) than Sheridan's original five (Plan, Teach, Monitor, Intervene, Learn): Teach is absorbed into Plan because plans are written directly in natural language the agent can act on, Monitor splits into Monitor and Wait, Intervene splits into Teach and Manual Fix, and Learn becomes the optional Update Assets.conf: high
extendsWhat's an Orchestrator—and Why Does Software Need One? · p2 — The paper gives the abstract claim that developers are 'converging into an orchestrator role' empirical structure: a concrete seven-stage breakdown of what that orchestration actually consists of, grounded in 19 developers' own workflow diagrams rather than a single practitioner's account.
supportsHuman judgment doesn't leave the software factory. It relocates. · p2 — This paper's empirically derived seven-stage structure -- judgment concentrated in Plan, then relocated across Review, Teach, Manual Fix, and Update Assets -- is a formal academic counterpart to Osmani's informally stated claim that human judgment doesn't leave the loop as agent autonomy increases, it relocates upstream and downstream.
explicitA newly identified 'Wait' stage, distinct from active Monitor, occurs when developers deliberately disengage attention from a running agent -- to work on other tasks, supervise other agents, or do something unrelated -- because they trust the agent to continue without observation for an extended period.conf: high
extendsHuman Oversight Is Not a Human in the Loop · p8 — The paper's finding that developers shift from active Monitor to disengaged Wait as trust in a plan grows is a concrete, practitioner-level instance of the general claim that the same nominal review position produces progressively weaker oversight as the volume and speed of agent-generated work grows.
explicit'Review' is a distinct stage from 'Monitor' -- unlike in Sheridan's original framework, where assessment is folded into monitoring -- because developers specifically assess the gap between their original intent and the agent's completed, bounded output, a gap that 'only becomes visible once you get to this stage.'conf: high
explicitcontestedSupervisory demands span multiple stages rather than being confined to any single one: three recurring demands -- keeping agent work aligned with developer intent, enforcing quality standards beyond task completion, and maintaining understanding of agent-produced work -- each surface in different forms across Plan, Monitor, Review, Teach, and Manual Fix.conf: high
contradictsWhat's an Orchestrator—and Why Does Software Need One? · p11 — This paper's empirically grounded 'maintaining understanding' demand -- developers actively reading agent reasoning and periodically redoing work themselves specifically to retain comprehension -- complicates the orchestrator piece's claim that shipping responsibly is compatible with not reading every line of generated code; the 19 developers studied here treat comprehension-preservation as necessary, ongoing work rather than an optional extra.
explicitcorroboratedDevelopers manage supervisory demands primarily by concentrating effort upfront in Plan: well-structured plans (explicit non-goals, 'change contracts' restricting which files may be touched, concrete examples/pseudocode of the desired result, written definitions of done) are associated with less Monitor, Teach, and Review work later, and let developers shift from active Monitor to disengaged Wait.conf: high
supports ←How Anthropic Teams Use Claude Code · p10 — Anthropic's own non-technical teams independently converge on the academic paper's core strategy of concentrating supervisory effort in planning: a separate, extensive planning phase (in Claude.ai) specifically intended to reduce how much correction and supervision implementation in Claude Code will later require.
explicitcorroboratedDevelopers distribute supervisory work in three distinct arrangements: delegating supervisory loops entirely to other agents or deterministic hooks, collaborating with agents on shared supervision (agent handles routine checks, human inspects what remains), and deliberately retaining some supervision themselves when delegating would cost more or would erode their own understanding of the system.conf: high
supports ←How Anthropic Teams Use Claude Code · p7 — Growth Marketing's explicit two-sub-agent design (a headline agent, a description agent) 'because this makes debugging easier' is a concrete instance of the paper's broader finding that developers distribute supervisory work by splitting it across agents with narrower, more reviewable scopes.
explicitDevelopers convert recurring supervisory corrections into persistent, reusable assets (linters, MISTAKES.md-style logs, CLAUDE.md-style instruction files, retrospective-generated skill files) specifically to prevent repeating the same correction in future tasks, producing what developers describe as a compounding reduction in supervisory burden across a project's lifetime.conf: high
supportsHuman Oversight Is Not a Human in the Loop · p10 — The 'Update Assets' stage -- externalizing lessons into persistent files like MISTAKES.md and CLAUDE.md that feed back into future Plan stages -- is a concrete practice-level answer to the theoretical requirement that a system depending on human oversight must itself preserve the context needed to make that oversight possible.
explicitHooks are preferred over written guidance for supervisory actions that must run reliably, because written guidance can be 'read, weighed against everything else, and quietly deprioritised once context fills up,' while 'a hook is code; it fires every time whether the model thinks it's relevant or not.'conf: high
explicitRepeated correction of agent drift can itself produce further drift ('scope drift over long sessions'), because an agent that has been redirected multiple times 'starts building on its own previous assumptions' across successive fixes -- correction is not a costless or self-terminating process.conf: high
supportsMeet the Scope Creep Kraken · p1 — The paper's 'scope drift over long sessions' finding -- repeated correction causing an agent to build on its own prior assumptions -- is a mechanism-level account of the same dynamic the scope-creep piece names: AI removes the staffing friction that used to constrain scope creep, so drift compounds across a session rather than resetting.
explicitEven when agents fully satisfy the stated task, developers continue to find implementations 'lazy, sloppy, repetitive, and overly defensive... even with solid guardrails and tests,' indicating that functional task completion and code quality are tracked as separate, independent supervisory criteria rather than one implying the other.conf: high
implicitDevelopers deliberately retain certain Manual Fix work -- 'tricky architectural decisions,' or periodically redoing 'one risky function or migration without the agent, just to keep the map in my head' -- not because an agent couldn't do it, but specifically to counteract the erosion of their own understanding that full delegation would otherwise cause, treating comprehension-maintenance itself as supervisory work with its own cost.conf: medium
supportsHuman Oversight Is Not a Human in the Loop · p4 — Developers periodically redoing a risky function themselves 'just to keep the map in my head' is a grassroots countermeasure to exactly the competence-erosion risk ('ironies of automation') that the oversight paper names as a structural effect of automation removing routine practice.
explicitAs agent autonomy increases, the appropriate design target for supporting developers shifts from reducing the effort of a single supervisory action toward reducing the cost of configuring and maintaining whole delegated-supervision arrangements (multiple watching agents, hooks, review pipelines, shared assets) across many tasks and sessions at once.conf: medium

Arguments

Developers should invest disproportionate effort in planning (Plan) rather than spreading equal supervisory effort across all seven stages.

Explicit premises:

Well-structured plans let developers shift from active Monitor to disengaged Wait, freeing attention for other agents or tasks
Stronger plans are associated with fewer repeated Teach cycles -- in the best case 'the whole thing can run in literally one pass, no looping needed'
Planning builds the developer's own understanding of the architecture before any code is delegated, which later supervisory stages (Review, Manual Fix) depend on

Implicit premises:

Supervisory effort invested early is more leveraged than the same effort invested after problems have already accumulated in the implementation
A developer's limited attention is the scarce resource being allocated across the seven stages, so moving cost from later, compounding stages to a single bounded Plan stage is a net efficiency gain

Inferences:

Systems and practices that help developers articulate, stress-test, and keep a plan visible throughout execution should yield outsized reductions in total supervisory cost compared to tools that only improve later-stage review or correction

Narratives

P18's account of why Review had to become its own stage: 'The plan cannot capture all of my intent -- some things get compressed, or it fills them in by guessing, and those only become visible once you get to this [Review] stage... What I look for in Review is the gap between my intent and the result.'
→ Grounds the structural claim that Review is a qualitatively different act from Monitor -- assessing a completed, bounded artifact against original intent, not watching an agent work in real time -- and explains why the framework departs from Sheridan's original single Monitor stage.
The MISTAKES.md pattern: one developer, 'tired of writing the same review comments,' built a static linter to catch the issue automatically before commit; another maintains a MISTAKES.md file where every agent mistake gets documented, paired with a CLAUDE.md rule instructing the agent to record them; a third runs a milestone-end retrospective where 'the retro edits or writes skill files and meta-learnings as memory.'
→ Illustrates concretely how the 'externalizing repeated supervision into assets' strategy is actually implemented day to day, and why developers describe the result as a 'compounding effect' rather than a one-off fix.

Questions addressed

✅ Can Sheridan's classical five-stage human-supervisory-control framework (built for automated systems like autopilots executing human-specified instructions) adequately describe how developers supervise AI coding agents, which determine their own approach to a delegated goal?
No, not without reconfiguration -- the 19-developer study shows Teach is absorbed into Plan, Monitor splits into Monitor and Wait, Intervene splits into Teach and Manual Fix, and Learn becomes an optional Update Assets stage that externalizes learning into persistent assets rather than only updating the supervisor's own understanding.
✅ Does the seven-stage framework generalize beyond the 19 observed lab sessions to the broader population of developers supervising AI coding agents in real-world use?
The paper applies the framework as an analytical lens to 102 Reddit threads (12,912 comments) from three developer subreddits spanning a 3-month window, and reports that the demands and strategies identified in the lab study also appear at scale in these naturalistic discussions.
⭕ Does greater investment in planning actually cause less supervisory work later, or do developers who already do less corrective work simply also happen to plan more (a confound of skill or task type)?
The paper's own §8.3 explicitly names testing 'whether greater effort in Plan reduces later correction' as future work, and acknowledges the current findings are a qualitative pattern across interviews and Reddit discussion rather than a controlled or causal test.
⭕ What happens to the seven-stage framework, and to developers' skills, agency, and sense of ownership, as agent autonomy continues to increase beyond what was observed in this study?

Gaps

incompleteThe workflow-diagramming method relied on retrospective reconstruction after a single ~40-minute task, and participants were introduced to Sheridan's five-stage framework before drawing their own diagrams -- a sequencing the authors themselves flag (§9) as a possible anchoring effect on how participants segmented their own supervision, even though they were explicitly told not to follow it.
Read next: A within- or between-subjects comparison of workflow diagrams elicited with versus without prior exposure to Sheridan's framework, to measure the anchoring effect directly rather than only naming it as a limitation.
incompleteThe three supervisory strategies (concentrating effort in Plan, distributing work, externalizing into assets) are presented as complementary, but the paper doesn't examine whether they trade off against each other within a single developer's limited time budget -- e.g. whether time spent building and maintaining MISTAKES.md-style assets displaces time that could have gone into planning, or whether developers who delegate review heavily to other agents plan less thoroughly because they expect agent-review to catch gaps.
Read next: A time-allocation study tracking how individual developers split effort across Plan, asset maintenance, and delegated-review configuration, to test whether the three strategies are additive or substitutive.
incompleteThe paper documents that developers retain some Manual Fix work and periodically redo tasks themselves specifically 'to keep the map in my head,' but doesn't examine whether this countermeasure actually holds up over longer timeframes than the ~40 minutes observed in the lab study -- i.e. whether comprehension of agent-produced systems measurably degrades for developers who increasingly shift to Wait and heavy delegation across many parallel tasks over weeks or months.
Read next: A longitudinal study (as the authors themselves call for in §9) tracking a developer's ability to explain or modify agent-produced code weeks or months after heavy multi-session delegation, to test whether comprehension debt accumulates despite the Manual Fix countermeasure this paper documents.
incompleteThe framework documents WHERE developers position themselves across seven stages, but doesn't examine whether that positioning constitutes genuine oversight in the stronger sense this corpus's extraction of 'Human oversight is not human-in-the-loop' describes -- whether developers in Review or Monitor actually have sufficient time, independence, and authority to meaningfully govern the outcome, versus merely occupying a formal checkpoint under increasing parallelism and throughput pressure.
Read next: Cross the seven-stage framework against the human-oversight-is-not-a-human-in-the-loop conditions (context, competence, time, independence, authority) to test which of the seven stages, under high parallelism, actually retain real governing power versus becoming nominal checkpoints.

Closes gaps elsewhere

What's an Orchestrator—and Why Does Software Need One? · g4 — This paper's 'Maintaining Understanding of Agent-Produced Work' demand (§7.1.3) supplies exactly the counter-position g4 flags as missing: experienced developers across 19 interviews and 102 Reddit threads describe reading agent reasoning, reviewing diffs, and periodically redoing risky work themselves specifically because understanding-without-reading is not sufficient to stay in control of a system they remain accountable for.