20 Loop Design Patterns Every AI Engineer Should Know

Rahul (@sairahul1) · tweet · 2026-07-08 Reader 🖥️

Agent Memory Agentic Loop Evals Feedback Loops Multi-Agent Orchestration Recursive Self-Improvement

Reliability and quality in production AI come not from bigger models or better prompts but from wrapping model calls in an improvement loop (generate -> evaluate -> learn -> adjust), and here is a taxonomy of ~20 such loops -- all of which, the thread finally concedes, reduce to one substrate: Act -> Observe -> Evaluate -> Adjust.

practical · expository — A catalog, not an argument: a numbered taxonomy of ~20 control-flow 'loop' patterns for agent systems, grouped into 5 categories (quality / memory / planning / exploration / system-optimization). Practical intent (build these), but engagement-bait genre: 'worth six figures', 'Save this', 'Repost', 'Follow @sairahul1'. Adler adapted for a catalog: unity carries the load; propositions capture each pattern; arguments are thin and mostly implicit.

Structure

Part	Role
Hook (agents vs loops)	Frames the thesis: an agent is a worker (single pass); a loop is what makes the worker improve. 'The teams shipping production AI are not writing better prompts. They are building better loops.'
Category 1 -- Quality Improvement Loops (patterns 1-5)	Generate-then-evaluate-then-revise patterns: separate a generator from an evaluator/critic and gate output on quality.
Category 2 -- Memory Loops (patterns 6-10)	Persist experience (failures, successes, lessons) and retrieve it so future runs are smarter; compress memory to stay usable.
Category 3 -- Planning Loops (patterns 11-15)	Adaptive, runtime replanning: treat the plan as revisable on contact with reality; decompose goals; monitor progress; run until constraints pass.
Category 4 -- Exploration Loops (patterns 16-18)	Try multiple paths (branch, tree-search, adversarial debate) and select the survivor.
Category 5 -- System Optimization Loops (patterns 19-20)	The loop improves the loop: optimize the prompt and the workflow itself against measured latency/cost/quality.
The pattern behind all 20 (+ full map + CTA)	Collapses everything to Act -> Observe -> Evaluate -> Adjust; recaps the map; adds share/follow/bookmark calls-to-action.

Key terms

Loop (vs Agent)
The thread's central rhetorical move: an 'agent' is a worker that does the job once (Prompt -> Response -> Done); a 'loop' is the wrapper that makes the worker improve across iterations (Generate -> Evaluate -> Learn -> Improve). The claimed source of production-grade AI value.
Reflexion
Pattern 6: agent fails -> analyzes why -> stores the lesson -> retries with that lesson in context. Described as 'the most important self-improvement pattern that exists.'
Judge Ensemble
Pattern 5: run the same output through multiple evaluators and aggregate scores; only high-consensus outputs advance -- averaging out single-judge noise.
Adversarial Critique
Pattern 4: the critic's only job is to break the answer, not improve it (what assumptions fail? what evidence is missing? where is this confidently wrong?); the best answer survives the attack.
Tree Search
Pattern 17: expand the most promising nodes, prune the weakest, go as deep as needed; 'computationally expensive but finds solutions single-pass calls cannot.' Explicitly frames Branch-and-Explore (16) as the depth-1 special case.
Act -> Observe -> Evaluate -> Adjust
The thread's closing reduction: the single structure every one of the 20 loops shares. Presented as 'the entire recipe.'

Propositions

explicitGenerate -> Critique -> Rewrite: a separate critic reviews generated output and the generator rewrites on that feedback until a quality threshold is met, because the model that generates is not the best judge of its own output.conf: high
supportsAgent Harness Engineering · p7 — Generate->Critique->Rewrite's core claim (the generator is not the best judge of its own output; a separate critic finds what it missed) is a concrete instance of the corpus principle that separating generation from evaluation into distinct agents beats self-evaluation because agents skew positive grading their own work. The thread's best-founded claim.
supportsHarness design for long-running application development · p3 — Same separate-evaluator claim, corroborated by the harness paper's finding that tuning a standalone skeptical evaluator is more tractable than making a generator critical of its own work.
explicitScore-and-Retry: generate, score against a measurable criterion, and retry if below threshold -- best when quality is objectively measurable (extraction accuracy, format compliance, factual correctness).conf: high
explicitcontestedMulti-Critic: use several *specialist* critics (correctness, style, safety, domain) each evaluating independently; output must satisfy all before it exits.conf: high
contradictsAgentic Code Review · p13 — Multi-Critic (and, by extension, Judge Ensemble p5 and Debate p18) assumes stacking independent model-critics removes blind spots. The corpus contests this directly: a closed loop of models trained on broadly the same data shares the same blind spots, so a confident consensus with no human in it is borrowed confidence -- 'both very sure and very wrong'. The load-bearing tension for a rigorous, anti-hype reader.
explicitAdversarial Critique: a critic whose sole job is to break the answer (find failed assumptions, missing evidence, confident wrongness); the generator then defends or rewrites, and the best answer survives.conf: high
explicitJudge Ensemble: run one output through multiple evaluators and aggregate; only high-consensus outputs advance, averaging out single-judge scoring noise.conf: high
explicitReflexion: on failure, the agent analyzes why it failed, stores the lesson, and retries with that lesson in context -- so each iteration is smarter than the last.conf: high
explicitMemory Update: after every task, store what decision was made, what the outcome was, and what would be done differently, so future runs inherit that knowledge -- the system at month 6 has read six months of its own history.conf: high
explicitError Library: store every failure and, before acting on a new task, search the library first; if a similar failure exists, apply the known fix before starting -- so the system stops making the same mistake twice.conf: high
explicitSuccess Pattern: store successes too (the approach, the context, what made it work) and retrieve them when facing similar tasks -- learn from wins, not just mistakes.conf: high
supportsHow to Actually Prompt Fable 5, Straight From the Team That Built It · p8 — Success Pattern (store successful approaches and what made them work, not just failures) is instantiated by the Fable-5 persistent-memory practice of recording 'confirmed approaches and why they mattered' alongside corrections -- the same experience-memory mechanism as a concrete engineering instruction.
explicitMemory Compression: after N items accumulate, compress many specific memories into fewer higher-level abstractions, because unlimited memory is unusable memory -- keeping context manageable and the system fast.conf: high
explicitdevelopedPlan -> Execute -> Replan: create a plan, execute a step, observe the outcome, update the plan, and continue -- treating the plan as a revisable spiral, not a fixed waterfall, because plans break on contact with reality.conf: high
refines ←The Anatomy of an Agent Harness · p10 — 20-loops p11 (Plan -> Execute -> Replan) is listed with no cost or tradeoff (that thread's g2 = no failure modes/costs per loop). Anatomy decision 2 (ReAct vs plan-and-execute) refines it by supplying the tradeoff and a number: ReAct interleaves reasoning and action every step (flexible, higher per-step cost); plan-and-execute separates planning from execution; LLMCompiler reports a 3.6x speedup over sequential ReAct.
explicitDynamic Workflow: the pipeline branches at runtime based on intermediate results (if output A run branch X, if B run branch Y, if C skip ahead) rather than following a fixed Step 1 -> 2 -> 3 sequence.conf: high
explicitGoal Decomposition: recursively break a large goal into subgoals, tasks, and steps until each unit is small enough to execute in one call.conf: high
explicitProgress Evaluation: every N steps, stop and ask 'are we actually getting closer to the goal?'; continue the strategy if yes, change strategy/tools/plan if no.conf: medium
explicitConstraint Satisfaction: keep running until every constraint (all business rules) is met; the output is not done until each rule passes.conf: medium
explicitBranch-and-Explore: explore several paths simultaneously rather than committing to one, then compare outcomes and keep the best branch.conf: high
explicitTree Search: expand the most promising nodes, prune the weakest, and keep exploring to arbitrary depth until a solution is found -- computationally expensive but reaches solutions single-pass calls cannot.conf: high
explicitDebate: two agents argue opposite positions on one topic, each round challenging assumptions and demanding evidence, so the final answer emerges through disagreement rather than agreement.conf: high
explicitPrompt Optimization: run a prompt on a test set, score every output, identify where it fails, rewrite it to fix those failures, and rerun -- so the prompt improves automatically without a human touching it ('the best prompts in production were not written, they were evolved').conf: high
explicitcorroboratedWorkflow Optimization: the system measures its own latency, cost, and quality per step and then modifies its own workflow (parallelize slow steps, downshift models where quality holds, add a critic when quality drops) -- 'the loop improves the loop', systems that redesign themselves.conf: medium
supports ←Don't train the model, evolve the harness · p4 — The proposer + keep-if-better outer loop is a concrete production instance of the Workflow-Optimization pattern (the system measures quality/cost and modifies its own workflow), and of Prompt/Config Optimization (p19) — supplying the real numbers that thread lacked.

Arguments

Production-grade AI reliability and quality come from loop design (improvement wrappers around model calls), not from bigger models or better prompts. strength: low

Explicit premises:

The most capable AI systems in production today are not single model calls; they are loops.
The model that generates is not the best judge of its own output; a separate critic finds what the generator missed.
These 20 loop patterns 'show up repeatedly in production AI systems.'

Implicit premises:

These patterns are empirically validated in production (asserted throughout; no benchmark, citation, named system, or metric is given).
The loops compose and nest cleanly (implied by the categories and by pattern 20's 'loop improves the loop', but no grammar for how they combine is provided).
More iteration monotonically improves output (ignores latency/token cost, over-optimization, reward-hacking the judge, infinite-retry, and memory drift).
Multiple model critics/judges have INDEPENDENT blind spots, so stacking them (Multi-Critic, Judge Ensemble, Debate) eliminates error -- an assumption the corpus directly contests (agentic-code-review#p13: a closed loop of models shares the same blind spots -> borrowed confidence).

Inferences:

Because agents fail on the first pass at task X, add loop Y and quality improves -- applied uniformly across all 20 without conditions.
Because separation of generation and evaluation beats self-evaluation, more/stacked evaluators are better (an unsupported extrapolation from a sound premise).

Narratives

The factory worker who does the job once vs. the worker who studies every mistake, rewrites the playbook, and gets 3% better every single shift.
→ Motivating analogy for the agent-vs-loop distinction; carries the emotional thrust of the thesis. Note: it is a rhetorical analogy, not evidence -- the '3% better every shift' figure is invented for effect.
The system in month 6 is not the same as the system in month 1 -- it has read six months of its own history.
→ Illustrates the Memory Update / experience-accumulation patterns as compounding improvement. Frames accumulated memory purely as gain, with no mention of drift or poisoning.
Too slow? Parallelize two steps. Too expensive? Replace a GPT-4 call with a smaller model where quality holds. Quality dropping? Add a critic.
→ Concretizes Workflow Optimization as a self-tuning control loop -- the closest the thread comes to an operational recipe.

Questions addressed

✅ What recurring control-flow structures do production agent systems use to improve output beyond a single model call?
It enumerates ~20, grouped into quality / memory / planning / exploration / system-optimization loops.
✅ Is there a common structure underneath all agentic improvement loops?
Yes: Act -> Observe -> Evaluate -> Adjust. The thread offers this as the single substrate of all 20 patterns.

Gaps

incompleteNo decision procedure. Twenty patterns, zero guidance on when to pick which. Each pattern carries only a 'Used for: [domains]' tag (naming application areas, not selection criteria, forces, or trade-offs). This is a pattern CATALOG, not a pattern LANGUAGE -- it has none of the GoF/Alexander connective tissue (Intent, Applicability, Consequences, forces that resolve into a choice). For a reader whose bar is 'a taxonomy without when-to-use-which is half a pattern language', this is the central omission.
Read next: Read against a real pattern-structure: Anthropic's harness paper (extract-anthropic-harness-design-long-running-application-development#p8) ties evaluator use to a concrete condition -- worth its cost only when the task sits beyond what the model does reliably solo -- exactly the selection criterion this thread lacks. Also extract-ai-workflow-patterns-real-unit-of-adoption-2026 (guidance on which workflows to automate first / how patterns chain).
incompleteNo failure modes or costs per loop. Only pattern 17 (Tree Search) names any cost ('computationally expensive'). Every loop here adds latency and token cost and can degrade: over-critique/thrash, reward-hacking the evaluator, infinite retry against an unsatisfiable threshold, and -- for the memory patterns -- drift and poisoning. The thread frames accumulated memory as pure monotonic gain.
Read next: extract-anthropic-harness-design-long-running-application-development#p8 (the evaluator becomes unnecessary overhead when the task is within the model's solo capability) and extract-turing-post-5-patterns-long-running-agents#p9 and #p20 (memory drift as a real production failure mode; persistent state makes behavioral drift a property of the system, not a bug) -- the exact risk the thread's Memory Loops ignore.
incompleteThe patterns are not orthogonal, and the thread never acknowledges the overlaps -- it even undercuts its own '20 distinct patterns' premise by collapsing all of them into one meta-pattern (Act -> Observe -> Evaluate -> Adjust) at the end. Branch-and-Explore (16) is explicitly a depth-1 Tree Search (17); Error Library (8) and Success Pattern (9) are the same store with opposite valence; Adversarial Critique (4) and Debate (18) are the same adversarial pressure; Progress Evaluation (14) is the observe step of Plan-Execute-Replan (11); Constraint Satisfaction (15) generalizes Score-and-Retry (2).
Read next: Dedupe to the ~8 underlying mechanisms (evaluate-and-revise, experience-memory, memory-compression, adaptive-planning, runtime-routing, decomposition, search-over-alternatives, meta-optimization); read the '20' as valence/depth/count/stance/level variants inflated to a shareable round number.
incompleteZero evidence that any of these loops work. The thread claims they 'show up repeatedly in production AI systems' and yield 'systems that get better every single day after deployment' with no benchmark, no citation, no named production system, and no metric. Named research patterns (Reflexion, Tree Search/MCTS, Debate, prompt optimization) are invoked without sources.
Read next: Demand evals; cross-read the harness papers that at least attempt to report data (extract-agent-harness-engineering, whose own gap g2 notes even the Terminal-Bench data point is under-specified). Weight the handful of patterns with genuine research lineage over the rest.
closed byDon't train the model, evolve the harness — Supplies the missing evidence for the self-optimization / workflow-optimization loop family specifically (p19/p20): a documented case where an automated 'add one mechanism, keep if it beats the current best' loop moved a frozen model from 0% to frontier parity at ~7x lower cost, with the honest boundary that gains compound then flatten. Evidences that pattern, not all 20.
incompleteComposition and nesting are asserted but never specified. Pattern 20 ('the loop improves the loop') and the five categories imply loops nest and chain, but there is no grammar for how -- which loop wraps which, where they conflict, or the orchestration cost of stacking them. For a systems thinker, the composition rules are exactly the missing part.
Read next: extract-anthropics-three-agent-harness#p8 (the handoff artifact between composed agents is the critical design variable) and extract-anthropic-harness-design-long-running-application-development#p10 (file-based inter-agent communication as sufficient orchestration) show what specifying composition actually looks like.
unsupportedMulti-evaluator patterns (Multi-Critic p3, Judge Ensemble p5, Debate p18) assume that stacking model-based critics eliminates blind spots -- an independence assumption that is stated nowhere and is directly contested: a closed loop of models trained on broadly the same data shares the same blind spots, so a confident consensus with no human in it is 'borrowed confidence' (very sure and very wrong).
Read next: extract-agentic-code-review#p13 (recorded here as a contradicts relation to p3). The corrective is a human or a genuinely out-of-distribution evaluator, not more same-family model-judges.
closed byUsing AI to write better code more slowly — That gap calls for 'a human or a genuinely out-of-distribution evaluator, not more same-family model-judges' as the corrective to stacked-critic 'borrowed confidence.' Lawson's actual workflow is exactly this: three genuinely different, cross-vendor reviewers (Claude sub-agent, OpenAI's Codex, Cursor's Bugbot) plus a mandatory independent human research pass before any finding is trusted — not a closed loop of same-family model-judges reaching unverified consensus.
incompleteSelection bias baked into the format. '20' is a round, shareable number, and the padding that reaches it (valence flips, depth variants, stance variants) is driven by thread virality -- reinforced by the 'worth six figures / Save this / Repost / Follow @sairahul1 / Bookmark this' framing -- not by the actual cardinality of distinct mechanisms.
Read next: Discount the count; treat the list as a prompt to name the ~5 patterns with real research lineage (Reflexion, Tree/MCTS search, multi-agent debate, automatic prompt optimization a la DSPy/APE, plan-execute-replan) and to source them properly.