Feedback Loops

Automated signals — tests, linters, type checks, AI reviews — that let humans and agents detect and correct errors quickly; their quality sets the ceiling on AI-assisted work.

Also known as: feedback loops, feedback loop, feedback loops (as ceiling), fast feedback, feedback speed / feedback loop, sensors (feedback controls), sensor, computational sensor, inferential sensor, computational controls, inferential controls, guides (feedforward controls), self-correction guidance, steering loop, Feedback Flywheel, double feedback loop, outrunning your headlights, observability, degree of automated verification

How authors use it

Oldest first (extraction date when no publication date is known).

AuthorTheir termTheir meaning
Presentation: AI-First Software Delivery: Balancing Innovation with Proven Practices
Wes Reisz · 2025-01-01
degree of automated verificationThe team's ability to automatically verify that generated code is correct, which depends heavily on domain knowledge and available test infrastructure.
Patterns for Reducing Friction in AI-Assisted Development
Rahul Garg · 2026-02-24
Feedback FlywheelA systematic process of harvesting learnings from AI interactions to improve the other four patterns over time
Harness engineering for coding agent users
Birgitta Böckeler · 2026-04-20
guides (feedforward controls)Controls that anticipate the agent's behaviour and steer it before it acts, increasing the probability of good first-attempt results. Examples: AGENTS.md, skills, code mods, LSP
Harness engineering for coding agent users
Birgitta Böckeler · 2026-04-20
sensors (feedback controls)Controls that observe after the agent acts and help it self-correct. Particularly powerful when signals are optimized for LLM consumption (e.g. custom linter messages with correction instructions)
Harness engineering for coding agent users
Birgitta Böckeler · 2026-04-20
computational controlsDeterministic, fast, CPU-run controls: tests, linters, type checkers, structural analysis. Milliseconds to seconds; reliable results
Harness engineering for coding agent users
Birgitta Böckeler · 2026-04-20
inferential controlsSemantic, probabilistic, GPU/NPU-run controls: AI code review, LLM-as-judge. Slower, more expensive, non-deterministic, but capable of richer semantic judgment
Harness engineering for coding agent users
Birgitta Böckeler · 2026-04-20
steering loopThe human practice of iterating on the harness by converting recurring agent failures into improved feedforward and feedback controls
Sam Hatoum's Post
Sam Hatoum · 2026-04-20
feedback speed / feedback loopThe elapsed time between expressing intent (writing a spec) and learning whether the resulting implementation is correct
It Ain't Broke: Why Software Fundamentals Matter More Than Ever
Matt Pocock (AI Hero conference talk, AI Engineer) · 2026-04-23
outrunning your headlightsProducing code faster than your feedback loops can validate it; the rate of feedback is your speed limit (from The Pragmatic Programmer)
[FULL WORKSHOP] AI Coding For Real Engineers - Matt Pocock, AI Hero
AI Engineer (channel); speaker: Matt Pocock (AI Hero) · 2026-04-24
feedback loops (as ceiling)Automated tests + type checks + lint + build that an agent can run autonomously after each change. Pocock asserts: 'the quality of your feedback loops influences how good your AI can code. Essentially, that is the ceiling.'
Fragments: May 5
Martin Fowler · 2026-05-05
double feedback loopTwo concurrent loops: (1) building/checking the product, (2) reshaping the tool/process itself when friction is felt
Maintainability sensors for coding agents
Birgitta Böckeler · 2026-05-19
Sensor (computational vs. inferential)A feedback mechanism on the path to production that gives the agent (and the human) information to reflect on and correct maintainability. Computational sensors are deterministic tools (ESLint, type checker, dependency-cruiser, mutation testing); inferential sensors are AI-judgment-based (a security-review prompt, a data-handling-review prompt). Some sensors are hybrid.
Maintainability sensors for coding agents
Birgitta Böckeler · 2026-05-19
Self-correction guidanceCustom lint/error messages engineered to instruct the agent on how to judge and act — described as 'a good kind of prompt injection' — as opposed to generic, human-oriented default tool messages.
Three more static code analysis sensors
Birgitta Böckeler (published on martinfowler.com) · 2026-05-20
sensorAny tool — deterministic or AI-driven — that feeds back signals about a code change to the agent (or human) so that self-correction is possible before issues reach human review. Sensors run at different cadences: in-session, in CI, on a slow schedule, or in production.
Three more static code analysis sensors
Birgitta Böckeler (published on martinfowler.com) · 2026-05-20
computational sensorA deterministic, rule-based tool whose output is reproducible given the same inputs (linter, type-checker, dependency-cruiser, coupling-analyser, SAST).
Three more static code analysis sensors
Birgitta Böckeler (published on martinfowler.com) · 2026-05-20
inferential sensorAn LLM-driven review (prompt + model + sometimes auxiliary CLI tools) whose output is interpretive and not bit-reproducible. Useful when 'what counts as bad' depends on context.
Three more static code analysis sensors
Birgitta Böckeler (published on martinfowler.com) · 2026-05-20
self-correction guidanceExtra prose embedded in a sensor's error/warning messages, written specifically so that an AI agent reading the output understands what to do — effectively a controlled prompt injection at the point of failure.
Shifting Platform Development from Projects to Products
Ben Linders · 2026-07-02
Feedback loopsSignals from actual platform users about usability and value, contrasted against internal sprint metrics (velocity, completion, quality) that the old model relied on.
Experiences with local models for coding
Birgitta Böckeler · 2026-07-08
sensor (for self-correction)A feedback channel the agent can call to check and correct its own output (e.g. a browser it can inspect after editing a chart). In the automated eval she gave the agent a browser as its only sensor, and it never called it; she argues expanding sensor use could enable self-correction and raise viability.
11 Tips For AI Coding With Ralph Wiggum
Matt Pocock · 2026-07-11
feedback loops (as guardrails)Types, unit tests, browser automation, linting and pre-commit hooks wired so that commits are blocked unless everything passes — the agent literally cannot declare victory on red. Their quality is the ceiling on the loop's output quality; 'the rate at which you can get feedback is your speed limit.'

Documents (58)

Propositions touching this concept (78)

By publication month, oldest first (extraction month when no publication date is known).

2018-07

2019-02

2021-01

2021-04

2024-04

2024-06

2025-02

2025-03

2025-06

2025-07

2025-10

2025-12

2026-01

2026-02

2026-03

2026-04

2026-05

2026-06

2026-07

2026-08

2026-09

2026-10

Tensions

Can harness engineering actually solve the AI code quality problem?
from: What is the right approach to building harnesses around AI coding agents, and where are the limits?
Assessment: This is the central divergence in the literature. Osmani is most optimistic, operating from benchmark evidence that harness changes produce dramatic improvements on coding tasks. Böckeler occupies the middle ground, distinguishing solvable from unsolved harness domains. Beck is most skeptical, questioning whether harnesses address the root cause at all. The disagreement is partly empirical (what do the benchmarks actually measure?) and partly about scope (Osmani focuses on task completion; Beck focuses on long-term code value including flexibility). They may not be contradicting each other so much as talking about different time horizons: harnesses help with immediate task success, but the tarpit emerges over accumulated changes.
What is the right metaphor for the human-AI relationship in coding?
from: What is the right approach to building harnesses around AI coding agents, and where are the limits?
Assessment: These metaphors are not mutually exclusive but they lead to different design decisions. Garg's teammate metaphor emphasizes collaboration rituals and may underplay the need for hard constraints (you don't sandbox a teammate). Böckeler's governor metaphor is the most theoretically rigorous but may overclaim the applicability of control theory to creative work. Osmani's engineering metaphor is the most actionable but may miss the social and organizational dimensions. Thoughtworks' organizational metaphor captures the broadest scope but is the hardest to operationalize.
Will better models make current harness practices obsolete?
from: What is the right approach to building harnesses around AI coding agents, and where are the limits?
Assessment: Osmani makes the strongest case for permanence with concrete evidence of scaffolding migration. Beck keeps the door open for the Bitter Lesson hypothesis. The truth likely depends on the type of harness: mechanical safeguards (sandboxes, permission controls) may always be needed, while knowledge-transfer patterns (AGENTS.md, priming docs) may become less necessary as models improve at in-context learning.
Is the model a swappable, commoditized component (so selection is a cost decision), or is model choice load-bearing (so selection is a capability decision)?
from: When working with AI coding agents, how should you select the model, its reasoning/effort level, and its context size for a given task?
Assessment: Both camps are right about different task strata, and the tie is broken by the routing camp's own artifact. Commoditization holds on the bulk/background tier (musistudio and the Pragmatic Engineer are correct that a cheap or open model suffices for a large share of tokens), but model choice is load-bearing exactly on the hard tier — the 'hardest 40%' the routing camp routes AWAY (the Pragmatic Engineer's own gap notes 60%-of-token-spend is not 60%-of-value). nrehiew's 6.5x over-editing spread and stellaraccident's collapse are measured on precisely that hard tier. The clincher the swap camp does not concede: musistudio's OWN per-model transformers (tooluse, enhancetool, reasoning, deepseek, gemini) prove the harness is NOT model-agnostic — every model needs bespoke request/response rewriting just to work behind Claude Code, so the 'any model' artifact self-refutes. Delangue even concedes the frontier gap himself. Verdict: select cheap-and-swappable for bulk, and treat the model as a load-bearing capability decision for the hardest slice — the two positions are not competitors but a partition of the task distribution.
Context size: should you shrink the problem to fit a small usable window (decompose / reset), or grow and trust a large window?
from: When working with AI coding agents, how should you select the model, its reasoning/effort level, and its context size for a given task?
Assessment: The apparent 'grow / trust the big window' position for CODING largely dissolves once retrieval is separated from coding — which is exactly Pocock's move ('good for retrieval, less good for coding') and musistudio's (longContext is a routed escalation, not the coding default). No text argues you should code inside a full 1M window. The only genuine pressure toward trusting a bigger window is the vendor-forward Fable 5 / Opus 4.6 claim that coherence loss is being engineered away — but that is a moving target (convergence 5), not a stable dial, and even the harness paper only relaxed context resets AFTER measuring that the new model no longer needed them. A secondary, live sub-dispute survives: reset vs compact. Pocock, the harness paper and best-practices favor clearing/resetting to a fresh agent; Hills and Osmani lean on compaction. The reset camp has the better argument on today's models — compaction preserves the same anxious agent — but the disagreement is real and unresolved. Verdict: shrink-and-decompose is the default for coding; grow only as an explicit retrieval escalation; prefer reset over compact unless the model version has demonstrably killed context anxiety.
Effort/reasoning: does more thinking make output better, or does more reasoning make it worse (over-engineering / over-editing)?
from: When working with AI coding agents, how should you select the model, its reasoning/effort level, and its context size for a given task?
Assessment: This looks like a head-on collision but is really the corpus resolving a category error, so I keep it as a divergence-under-adjudication rather than a live two-camp fight. The two things bundled into one 'effort' dial are separable: reasoning DEPTH (deliberation and research before acting — what stellaraccident measures as the Read:Edit ratio) versus action SCOPE (how much the model rewrites — what nrehiew measures as over-editing). They are separable but DEFAULT-COUPLED: more reasoning, unconstrained, tends to produce more scope. nrehiew is the empirical bridge — constrained reasoning models are the MOST faithful, proving depth and tight scope combine positively when you address them separately. Fable 5 already acts on this, pairing 'reduce effort' with a distinct anti-over-engineering instruction. The actionable verdict: do NOT cut reasoning depth to fix over-editing (that is stellaraccident's exact grievance and it backfires into shallow, edit-first behavior); keep the depth and add an explicit scope constraint. The failure is treating one knob as if it controls both correctness-depth and edit-scope.
Who selects the model/effort/context — a human deciding per task, or an automatic router — and who is trusted to hold the dial?
from: When working with AI coding agents, how should you select the model, its reasoning/effort level, and its context size for a given task?
Assessment: The human-vs-auto axis is less about capability than about (a) whether task difficulty can be detected a priori and (b) who bears the principal-agent risk. No router in the corpus actually measures difficulty — musistudio routes by token-count and static scenario, and the Pragmatic Engineer's routers infer 'good enough' without a demonstrated oracle — so 'auto' today means 'static-heuristic', not 'optimal'. Automation is safe when the user owns the policy (musistudio) and dangerous when the router profits from the choice (Cursor Auto) or when the provider adjusts the dial invisibly (stellaraccident). Verdict: keep the human holding the difficulty judgment and the effort dial until routers can both measure adequacy and prove neutral incentives; automate only the mechanical dispatch, over rules you own and can audit.

Open gaps (125)