Software Factories, Light and Dark
AddyOsmani.com ยท rss ยท 2026-07-20 source โ Reader ๐ฅ๏ธ
Agent HarnessAgentic LoopBack Pressure (Verification-Bound Autonomy)Cognitive DebtDark Factory (Fully Automated Software Pipeline)Human as Orchestrator of AgentsHuman-in-the-LoopLoop EngineeringSoftware FactoryVerifiability (as the Quality Bar)
Because verification, not generation, is the real bottleneck of an AI-driven 'software factory,' an engineer's central job is to decide loop by loop how much autonomy each part of the pipeline can earn โ full automation ('dark') only where a check is cheap, fast, and hard to fake, human judgment ('lit') everywhere else โ with the human's own role relocating from writing code to owning and verifying that outer decision.
practical ยท mixed โ Builds a theoretical vocabulary (loop/harness/factory, dark/lit, back pressure) but the piece is ultimately practical: it culminates in an imperative ('your hardest job now is knowing which checks to build and how much autonomy to delegate') and a checkable decision procedure for where to put 'the lights.' Expository definition sections alternate with argumentative sections and two concrete narratives (Dex's four-month dark-factory run; the nightly lint-fix cron), so it reads as mixed rather than purely argumentative or expository.
Structure
| Part | Role |
|---|---|
| Historical framing: Bob Bemer's 1968 'software factory' dream and Dex Horthy's AI Engineer World's Fair talk | context/motivation โ establishes the idea is old, has failed before, and argues why the last two years changed the calculus |
| The loop/harness/factory definitional stack ('the loop is the atom, the factory is the loop at scale') | conceptual scaffolding โ three nested terms the rest of the argument is built on |
| The factory diagram: queue -> harness builds -> automated checks -> review gate -> deploy -> monitoring -> signals | model/evidence โ identifies the review gate as the single non-scaling, expensive box in an otherwise near-zero-cost pipeline |
| Dark factory definition, the manufacturing analogy (FANUC, Xiaomi), comprehension debt, and Dex's four-month fully-automated run | evidence โ a narrative case for the risk of full automation, showing the failure mode is real and discovered late rather than gradually |
| 'The bottleneck was never generation': back pressure defined, the funnel image, and why better models don't close the gap | central argument โ names the constraint and explains why it resists being solved by model improvement alone |
| Lit factory prescription: judgment moved upstream to design/architecture, and ordinary architectural practice reframed as an agent safety net | practical conclusion โ the prescription that follows from the back-pressure argument |
| 'What earns a loop the dark': operational criteria (cheap, frequent, hard-to-fake, immediate, short) for granting full autonomy | operationalization โ turns the back-pressure rule into a checkable decision procedure |
| 'Loops, graphs or state machines?': the argument that agentic autonomy is really walking a predefined graph, illustrated with a bug-fix example | secondary argument โ a recommended implementation pattern for making back pressure legible |
| 'Where the human actually goes': the human relocates to owning the outer loop (decide, verify, approve, own consequences) | conclusion โ resolves the question of what remains for engineers to do |
Key terms
- Loop
The smallest unit of agentic work: one agent gathering context, taking an action, checking the result, and repeating until some condition is met. Everything above it โ harness, factory โ is loops stacked on loops. - Harness Agent Harness
The walls around a loop: the sandbox it runs in, the tools it can reach, the memory that survives between runs, and the gates that decide what 'done' means. The loop is the behavior; the harness is the environment. - Software factory Software Factory
Many harnessed loops running at once, fed by a queue of work and drained through a review gate into production, with humans owning the whole thing from above โ explicitly 'not a bigger agent' but 'an org chart made of loops.' - Dark factory Dark Factory (Fully Automated Software Pipeline)
A software factory that runs with no human reading step at all โ code ships verified only by other machines, borrowed directly from manufacturing's 'lights-out' automation (FANUC since 2001, Xiaomi in 2024). - Lit factory
The same automated pipeline as a dark factory, but with human judgment kept 'on' wherever a wrong call is expensive โ crucially, judgment is moved upstream to design/product/architecture decisions, not merely re-added as end-of-pipeline code review. - Comprehension debt Cognitive Debt
The widening gap between how much code exists and how much any human still understands. A dark factory doesn't pay this down โ it accumulates it as fast as possible, with tests staying green the whole time. - Back pressure Back Pressure (Verification-Bound Autonomy)
The rule that a loop can be handed only as much autonomy as can be cheaply and reliably verified, 'and not one inch more.' Verification, not generation, is the real constraint on how fast a factory can run. - Outer loop / inner loop (as applied to the human)
The inner loop is what the agent executes: investigate, diagnose, implement, run tests, write a report. The outer loop is what the human now owns: decide whether it's the right problem to address, verify the diagnosis and implementation are sound, approve the change, and carry the consequences of being wrong.
Propositions
- explicitdevelopedA software factory is many harnessed agentic loops running at once, fed by a queue of work and drained through a review gate into production, with humans owning the whole system from above โ an org chart made of loops, not a bigger single agent.conf: highrefinesLooking back at the Future of Software Engineering unconference ยท p1 โ Gives the 'software factory' endpoint a concrete queue -> harness -> review-gate -> deploy wiring diagram and an 'org chart made of loops' framing, sharpening the vaguer 'self-evolving system that builds a feature into the existing system' definition.extends โCloudflare Introduces the Agent Development Lifecycle to Replace Traditional SDLC ยท p1 โ Names the same 'automated software factory' end state and adds Cloudflare's specific infrastructure answer (Workflows, observability, Agent Access Model) to the general definition given there.extends โInside a Software Factory ยท p1 โ Both articles quote Osmani's 'org chart made of loops' framing of a software factory; Iusztin's eight-stage/three-bucket taxonomy is a more granular, SDLC-mapped elaboration of the same underlying model.extends โThe New SDLC With Vibe Coding: From ad-hoc prompting to Agentic Engineering ยท p6 โ Same author's own earlier, more detailed framing of the factory model ('harnessed agentic loops... an org chart made of loops') restated here as the organizing mental model for the whole SDLC paper.
- explicitA loop is the smallest unit of agentic work โ one agent gathering context, acting, checking the result, and repeating until a condition is met โ and everything above it (harness, factory) is loops stacked on loops.conf: high
- explicitA harness is the environment around a loop (sandbox, tools, memory, done-gates) that makes an otherwise-unbounded model useful and safe to run; a raw model with no harness will spin forever.conf: high
- explicitIn the factory's closed-loop diagram, every stage is near-zero cost (generation, tests, static analysis, scanning) except the review gate, where human judgment lives โ the sole expensive, non-scaling box in the pipeline.conf: high
- explicitA dark factory is a fully automated software pipeline in which no human reads the code that ships, verified only by other machines, directly borrowing the 'lights-out manufacturing' precedent (FANUC since 2001, Xiaomi in 2024).conf: high
- explicitcorroboratedRemoving the human review step makes a team's perceived throughput jump dramatically, which is exactly what makes dark factories deceptively easy to adopt at first, while their real costs stay hidden.conf: highsupports โReinventing team flow within AI uncertainty ยท p3 โ Both describe the same mechanism from opposite ends: Osmani notes that removing the human review step makes perceived throughput jump; this piece gives the concrete downstream consequence โ a faster fix cycle just clogs the unchanged pull-request approval queue, a local win that doesn't improve the system.
- explicitcorroboratedComprehension debt (the growing gap between how much code exists and how much any human still understands) is not paid down by a dark factory; it is taken on as fast as possible, with tests green the whole way.conf: highextendsLoop Engineering ยท p12 โ Same author's earlier claim that comprehension debt grows with unread loop output is generalized here into the full dark/lit factory taxonomy and explicit criteria for when a loop is allowed to stay unread.supports โHuman judgment doesn't leave the software factory. It relocates. ยท p7 โ Both pieces describe comprehension/cognitive debt as the real, hidden cost of agentic throughput; this piece supplies first-person evidence (the 'feature I had to relearn' story) for the sibling's more abstract claim that comprehension debt accumulates even while tests stay green.
- explicitModel-only automated coding fails specifically against comprehension debt in complex, brownfield, long-lived production systems (not in weekend/toy projects), because a decade-old enterprise codebase has to be maintained at a professional pace that a few months of unread development cycles can't survive.conf: high
- explicitcorroboratedRunning a fully automated code factory with zero human reading of output for about four months produced a failure severe enough to require painstaking manual debugging just to locate โ evidence the failure mode is real, not hypothetical, and surfaces 'quiet and late' rather than as a dramatic collapse.conf: highsupports โInside a Software Factory ยท p10 โ Both report a real personal failure of a fully autonomous, human-out-of-the-loop pipeline that went undetected/uncontrollable for a period before being caught โ Osmani's account is more concrete (about four months, a painstaking manual debug) and can be read as corroborating detail for Iusztin's vaguer 'something went offscript' story.
- explicitestablishedBack pressure โ you can only hand a loop as much autonomy as you can cheaply and reliably verify, 'and not one inch more' โ makes verification, not generation, the actual constraint on how fast a software factory can run.conf: highextendsLoop Engineering ยท p11 โ Turns the prior post's observation that verification stays entirely on the human into a named systems-thinking rule ('back pressure') plus concrete, checkable criteria for how much autonomy verification capacity can support.refines โHuman Oversight Is Not a Human in the Loop ยท p6 โ Osmani's back-pressure framing treats verification-boundedness as an engineering constraint on autonomy; Bataller's 'authority must be real' supplies the organizational/human-factors half โ the boundary must also be organizationally enforceable, not just technically defined.supports โThe New SDLC With Vibe Coding: From ad-hoc prompting to Agentic Engineering ยท p7 โ Both pin the real constraint on how fast AI-assisted development can run to verification/harness design, not model capability or raw generation speed ('back pressure').extends โHuman judgment doesn't leave the software factory. It relocates. ยท p3 โ The 'verification budget' (fast checks early, expensive checks reserved for later gates) is a concrete operational tactic for exactly the 'back pressure' constraint the sibling essay states abstractly: you can only grant as much autonomy as you can cheaply and reliably verify.supports โ#ai #programming #softwareengineering | Addy Osmani | 11 comments ยท p6 โ This post's 'generation is solved, verification is the new bottleneck' claim is the seed of Osmani's later, fully worked-out 'back pressure' rule (autonomy only as far as cheap, reliable verification allows).supports โOwn the Outer Loop ยท p11 โ Reaffirms, nearly verbatim, the earlier essay's back-pressure/autonomy-sizing claim.
- explicitImproving the underlying model does not close the generation/verification gap, because architectural-quality judgments have no fast, 'tidy' training signal โ the cost functions measuring architectural excellence play out over months or years, not seconds.conf: high
- explicitcorroboratedA lit factory keeps the same automated pipeline as a dark factory but relocates human judgment upstream โ into product, design, and architecture decisions before an agent starts a loop โ rather than only re-adding review at the end.conf: highsupports โInside a Software Factory ยท p5 โ Both claim the 'lit'/disciplined factory relocates human judgment upstream into brainstorm/plan (product, design, architecture) rather than only at a final review gate โ Iusztin adds the concrete claim that the human also returns at the very end (PR review/merge).extends โHuman judgment doesn't leave the software factory. It relocates. ยท p2 โ This piece restates and broadens the sibling essay's 'lit factory relocates judgment upstream' claim into a fuller principle -- judgment relocates both upstream AND downstream (to evidence, risk, ownership) -- and gives it the piece's own name: 'human judgment is being relocated.'
- explicitcorroboratedReviewing a short, upfront design/architecture plan is cheaper and more effective than reviewing the much larger diff it eventually generates, because it catches expensive, long-lived mistakes before their cost compounds.conf: highsupports โ#ai #programming #softwareengineering | Addy Osmani | 11 comments ยท p5 โ Both identify the specification/upfront design step as the real leverage point โ cheaper to get right than reviewing the large diff it eventually produces.
- explicitOrdinary, long-known architectural practices (strong types/signatures, test seams, legible layout, short call stacks, well-bounded component boundaries, dependency injection) now do a 'second job' as a cheap, hard-to-fake safety net against agent mistakes, precisely because the model itself won't supply that discipline.conf: high
- explicitdevelopedA loop earns fully automated ('dark') status only if its check is cheap, runs at high frequency, and relies on an oracle that can't easily be faked out and answers immediately without drifting over time (green/red tests, type gates, property tests, or a review agent with a real rubric).conf: highextendsAgentic Code Review ยท p7 โ Adds oracle-quality criteria (cheap, frequent, hard to fake, immediate, short-loop) alongside blast-radius/lifetime/audience as inputs to how much review or automation a change should get.extendsWhat Survived Contact ยท p13 โ Generalizes the single 'live-referenced acceptance oracle' case into a broader taxonomy (green/red tests, type gates, property tests, rubric-based review agents) of what counts as a trustworthy, hard-to-fake automated check.extends โHuman judgment doesn't leave the software factory. It relocates. ยท p5 โ The 'when green is misleading' anecdote (an agent satisfying a test's literal assertion rather than its intent) is a concrete failure mode of the oracle-fakability criterion the sibling piece names abstractly as a requirement for a check to earn fully-automated ('dark') status.
- explicitShort agentic loops are easier to verify than long ones because context accumulation causes an agent to lose the thread โ reliably useful for roughly three to ten steps, breaking down past twenty โ so loop length is itself a proxy for verifiability, and thus for automatability.conf: high
- explicitSetting every loop's switch to the same mode fails in both directions: 'all dark' risks a catastrophic reckoning months later, while 'all lit' creates a bottleneck that stalls the whole factory; the skilled work is deciding, loop by loop, where each switch goes.conf: high
- explicitAgentic autonomy is, underneath, always just walking a predefined directed graph โ any code can already be expressed as a control-flow graph โ so the 'genuinely new' move of the pure-loop era was throwing that structure away, not discovering it, and the industry's current pivot back to explicit graphs/state machines is a rediscovery of structure software always had.conf: high
- explicitEncoding an agent's work as an explicit graph, rather than a free-running loop, is back pressure made visible as a diagram: it trades some of the agent's freedom for mandatory checkpoints and legible failure points, so a broken run can be traced to the specific node that killed it.conf: high
- explicitdevelopedAs agentic work industrializes, the human's job moves from writing code to owning the 'outer loop' โ deciding whether a proposed fix addresses the right problem, verifying the diagnosis and implementation are sound, approving the change, and bearing the consequences of being wrong โ while the agent executes the 'inner loop' of investigation, implementation, and testing.conf: highextends โOwn the Outer Loop ยท p1 โ Elevates the earlier essay's single proposition โ the human's job moves from writing code to owning the 'outer loop' โ into a full three-term accountability framework (Quality, Verdict, Answerability).
- implicitcorroboratedSoftware engineering expertise is being redefined away from code-writing skill and toward judgment about verification design โ deciding what to check, how cheaply, and how much autonomy that check can support.conf: highsupports โ#ai #programming #softwareengineering | Addy Osmani | 11 comments ยท p4 โ The post's claim that SWE skill relocates to spec/verification is the same redefinition Osmani later develops explicitly as 'expertise redefined away from code-writing and toward judgment about verification design.'
- implicitThe review-gate bottleneck is a structural feature of any software factory, not an artifact of current tooling immaturity โ it can be relocated (upstream to design, or distributed into cheap automated oracles) but never eliminated.conf: medium
- implicitThe article treats verification and generation as separable, sequential stages (generate, then verify) rather than as something that could be redesigned to be co-produced โ it assumes the funnel shape (wide generation mouth, narrow verification neck) is a fixed given of the domain, not a possibly-contingent artifact of today's tooling.conf: medium
- implicitThe framework assumes a stable mapping from 'type of decision' to 'who should own it' (design = human, lint-fix = machine) rather than a boundary that should be continuously re-earned as verification capacity itself expands โ the dark/lit switch is treated as something you set, not something you keep re-running the calculation on.conf: medium
Arguments
A fully automated ('dark') software factory will eventually fail catastrophically, and the failure will be discovered late rather than gradually. strength: high
Explicit premises:
- Comprehension debt is the gap between code that exists and code humans understand.
- A dark factory has no human reading step, so nothing pays that debt down.
- Tests can stay green while comprehension debt grows, because tests measure behavior, not human understanding.
- Dex's four-month fully-dark run required painstaking manual debugging to even locate the failure it produced.
Implicit premises:
- Passing tests is not sufficient evidence that a system remains safe to keep changing without human comprehension โ test coverage undersamples the space of things that can go silently wrong architecturally.
- Human comprehension is a scarce resource that, once lost, cannot be cheaply reconstructed after the fact โ re-understanding a system nobody read is far more expensive than reading it as it was written.
- Enterprise/brownfield systems compound this risk more than small ones, because they are maintained far longer than 'weekend toys.'
Inferences:
- A team measuring 'AI productivity' primarily via velocity or lines-shipped, without a comprehension metric, cannot detect this failure mode until it is already expensive.
- Since the failure is discovered 'quiet and late,' this argues for continuous minimum-viable human reading rather than periodic audits โ but the article never draws this operational conclusion itself.
Autonomy should scale with verification capacity, not with generation capacity or model quality โ 'back pressure.' strength: medium
Explicit premises:
- Generation (code, tests, scanning) is near-zero marginal cost at scale.
- The review gate is the one box that doesn't scale for free โ it consumes finite human judgment.
- A surplus of bad PRs, not a shortage of generated code, is the actual complaint when volume outruns trustworthy gates.
- Improving the model doesn't close the gap, because architectural-quality judgments lack fast, 'tidy' training signal.
Implicit premises:
- Verification is fundamentally rate-limited by something that doesn't scale with compute โ human attention/judgment, or narrow automatable proxies for it โ so throwing more model capacity at generation can only widen, not narrow, the gap.
- A check only counts as 'verification' if it's a reliable proxy for the human's actual concern โ an easily-gamed check (e.g., merely 'tests green') doesn't count, even though it superficially looks like verification.
Inferences:
- Vendor productivity claims that report only generation-side metrics (tokens, PRs, or lines per day) measure the wrong side of the funnel, and could even be anti-correlated with real throughput once bad-PR cleanup cost is included โ an inference the article's own logic supports but never states.
Whether a given loop should run dark or lit is a decidable, criteria-based judgment, not a vibe or a blanket policy. strength: high
Explicit premises:
- The check must be cheap and run at high frequency.
- The oracle must be hard to fake out and must answer immediately without drifting.
- Short loops (roughly three to ten steps) are more reliably verifiable than long ones, because of context accumulation.
- High-stakes decisions (auth, billing, public API contracts, large blast radius, year-plus consequences) require a human, because only a person can catch certain classes of wrong answer.
Implicit premises:
- 'Cheap to verify' and 'safe to automate' are treated as equivalent โ the article assumes there's no case where a loop is cheap to verify yet still too risky to run dark (or vice versa: expensive to verify yet actually safe).
- The criteria are assumed to generalize across arbitrarily different domains โ the same taxonomy of 'oracle quality' is assumed to apply whether the change is a lint fix or a billing-engine change.
Inferences:
- Because the criteria are properties of the check, not of the code, the same underlying change could correctly be dark in one context (behind a strong, fast oracle) and lit in another (weaker oracle) โ autonomy is a property of the verification setup, not of the artifact itself.
The right way to build reliable agentic systems is to explicitly encode them as directed graphs/state machines with conditional edges, not as an unconstrained loop where the model picks its own path. strength: medium
Explicit premises:
- Any code can already be expressed as a control-flow graph.
- The 'liberation' of throwing the diagram away and letting the model pick tool-by-tool was the actual novel move of the pure-loop era, not a return to structure.
- Encoding work as a graph gives back pressure a visible shape โ mandatory checkpoints and legible failure points, so a broken run can be traced to a specific node.
Implicit premises:
- Legibility of failure (being able to point at the node that killed a run) is treated as an unqualified good, without weighing whether it costs the flexibility that made free-running loops valuable for novel or unanticipated cases in the first place.
- The convergence of multiple independent projects (LangGraph, LlamaIndex Workflows, David Khourshid's state-machine framing) on graph-based structure is taken as evidence of a correct architecture, rather than, e.g., a shared overcorrection or shared tooling-vendor incentive.
Inferences:
- This argument sits in some tension with the piece's own worry (Argument 1) about brownfield complexity requiring more autonomy-limiting discipline, not less โ if graphs constrain agents to sanctioned paths, that should reduce, not just relocate, the comprehension-debt risk the piece spends most of its length worrying about; the article doesn't connect these two threads explicitly.
Narratives
- Dex Horthy ran a fully automated code factory for about four months with no human reading any of the code it produced, and the resulting failure was severe enough that it took painstaking manual debugging just to locate.
โ Grounds the abstract comprehension-debt claim in a concrete, first-hand case โ supplies the evidence for Argument 1 and for the proposition that dark-factory failures surface 'quiet and late' rather than dramatically. - FANUC's lights-out manufacturing facilities (running since 2001) and Xiaomi's 2024 automated dark factory โ physical factories where robots assemble and ship product with no human ever inspecting it.
โ Grounds the 'dark factory' metaphor in physical precedent, making the software claim a literal analogy ('the floor is the diff') rather than just a rhetorical flourish. - Horthy's nightly GitHub Actions cron job that fixes exactly one anti-pattern or lint violation, commits, and opens a single small pull request unattended, so the team wakes up to a slightly better codebase and a diff short enough to read.
โ Supplies a worked, positive example of a loop that legitimately earns dark/automated status โ small blast radius, short, cheap and fast to verify โ illustrating the criteria in Argument 3. - The same bug-fix task walked through twice: once as a pure loop (figure out what's wrong, change code, run tests, repeat until it works) and once as a graph (reproduce, find cause, try fix, run tests, route a failing run back to the fix while only an approved passing run reaches done).
โ Makes the abstract loop-vs-graph distinction concrete, and shows exactly where back pressure enters as diagram structure โ supports Argument 4. - A two-hundred-line upfront design plan, read and approved before implementation, contrasted with the two-thousand-line generated diff a reviewer would otherwise have had to reverse-engineer the same decision from after the fact.
โ Illustrates why moving review upstream is cheaper โ an anecdotal size/effort comparison supporting the proposition that upfront review beats after-the-fact diff review.
Questions addressed
- โ
What, if anything, is genuinely new about the ~60-year-old 'software factory' dream now that AI coding agents exist?
What's new isn't the dream itself but that generation, tests, and scanning have become near-zero-cost, which makes the review gate visible as the sole remaining bottleneck โ the loop/harness/factory stack and the back-pressure framing are offered as the precise articulation of what changed. - โ
How much autonomy should a given automated loop be given?
The back-pressure rule plus concrete criteria: cheap, high-frequency, hard-to-fake, immediately-answering, non-drifting oracle, and short (3-10 step) loop length. - โ
Where does the human's job go once agents do most of the code-writing?
To owning the outer loop: deciding whether the right problem is being addressed, verifying the diagnosis and implementation, approving the change, and bearing the consequences of being wrong. - โ
Should agentic systems be built as free-running loops or as explicit graphs/state machines?
Argues for graphs/state machines as the way to make back pressure legible โ though see the gap below on when a free-running loop's flexibility might actually be preferable. - โญ How do you actually determine, in advance, whether a given oracle 'can't be easily faked out'?
- โญ What happens to teams that don't yet have the 'ordinary architectural practices' (types, seams, dependency injection, bounded components) that the lit-factory safety net depends on?
Gaps
- incompleteThe 'hard-to-fake-out oracle' criterion is asserted as a checkable property but never operationalized โ no method is given for determining, before deploying a loop dark, whether its check can or can't be gamed by an agent optimizing against it, even though the piece itself gestures at 'manufactured defects' as a real risk when volume outruns trustworthy gates.
Read next: Read Goodhart's-law / reward-hacking material and the corpus's reference-oracle discussions (e.g. extract-what-survived-contact) for how to detect a drifted or gameable oracle before trusting it. - incompleteThe 'lit factory' safety net (types, seams, legible layout, bounded components, dependency injection) is presented as something teams 'have always talked about,' but the article never addresses teams that don't already have it โ there's no stated path from a weak-architecture brownfield codebase to the lit-factory's prerequisites, leaving exactly the population most at risk of comprehension debt without a remediation route. This echoes an already-flagged gap elsewhere in the corpus about the same unstated-prerequisite problem for 'software factory' claims generally.
Read next: Read on legacy modernization and evolutionary architecture for how under-resourced teams retrofit the architectural prerequisites (types, seams, boundaries) the lit-factory model assumes. - illogicalThe piece argues both that free-running loops are dangerously unstructured (motivating the pivot to explicit graphs) and that short, low-stakes loops (the nightly lint-fix cron) legitimately earn full autonomy without any explicit graph structure โ it never reconciles when a loop needs graph-discipline versus when it's fine to run as a pure loop; length and stakes are offered as the answer for automation-worthiness, but that criterion is never explicitly connected back to the separate loop-vs-graph argument.
Read next: Read for a unified framework connecting loop-length/stakes-based autonomy criteria with graph/state-machine architecture choices, rather than treating them as two independent recommendations. - incompleteThe rule of thumb that 'an agent holds up for three to ten steps, then starts losing the thread past twenty' is stated (attributed to Dex Horthy) without any grounding for why that specific numeric range holds, what model or context window it was measured against, or whether it's expected to shift as context windows and context-management techniques improve โ it's treated as a durable constant rather than a fast-moving, model-dependent empirical measurement.
Read next: Look for benchmark data on agent step-count reliability across model generations and context-management techniques, to see whether this threshold is already stale. - illogicalThe closing note that 'Pangram scored this article as 100% human written' is offered as a credibility signal, but the piece doesn't examine (or seem to notice) that using an AI-detection tool's green/red verdict as the evidentiary bar for 'a human genuinely understood and is accountable for this text' is exactly the category error the whole essay warns against for code โ a cheap, automatable check standing in for real comprehension and accountability.
Read next: None needed โ this is a self-referential observation about the piece's own closing move rather than a pointer to further reading.
Closes gaps elsewhere
- We've Stopped Arguing About Whether AI Changes Software ยท g1 โ Supplies the concrete method the earlier piece named but didn't specify for 'is it verifiable enough': green/red oracles, type gates, property tests, and rubric-backed review agents, plus loop length (3-10 steps) as an operational proxy for verifiability.
Related documents
- Loop Engineering: Building Blocks, Adoption, and Impact
- Own the Outer Loop
- What the Hell Is a Loop, Anyway?
- #ai #programming #softwareengineering | Addy Osmani | 11 comments
- What is โloop engineering?โ
- 11 Tips For AI Coding With Ralph Wiggum
- 12-Factor Agents: Patterns of reliable LLM applications โ Dex Horthy, HumanLayer
- How we Claude Code
- Inside a Software Factory
- The Future of Work by Henrik Kniberg