Software Factories, Light and Dark

AddyOsmani.com ยท rss ยท 2026-07-20 source โ†— Reader ๐Ÿ–ฅ๏ธ

Agent HarnessAgentic LoopBack Pressure (Verification-Bound Autonomy)Cognitive DebtDark Factory (Fully Automated Software Pipeline)Human as Orchestrator of AgentsHuman-in-the-LoopLoop EngineeringSoftware FactoryVerifiability (as the Quality Bar)
Because verification, not generation, is the real bottleneck of an AI-driven 'software factory,' an engineer's central job is to decide loop by loop how much autonomy each part of the pipeline can earn โ€” full automation ('dark') only where a check is cheap, fast, and hard to fake, human judgment ('lit') everywhere else โ€” with the human's own role relocating from writing code to owning and verifying that outer decision.

practical ยท mixed โ€” Builds a theoretical vocabulary (loop/harness/factory, dark/lit, back pressure) but the piece is ultimately practical: it culminates in an imperative ('your hardest job now is knowing which checks to build and how much autonomy to delegate') and a checkable decision procedure for where to put 'the lights.' Expository definition sections alternate with argumentative sections and two concrete narratives (Dex's four-month dark-factory run; the nightly lint-fix cron), so it reads as mixed rather than purely argumentative or expository.

Structure

PartRole
Historical framing: Bob Bemer's 1968 'software factory' dream and Dex Horthy's AI Engineer World's Fair talkcontext/motivation โ€” establishes the idea is old, has failed before, and argues why the last two years changed the calculus
The loop/harness/factory definitional stack ('the loop is the atom, the factory is the loop at scale')conceptual scaffolding โ€” three nested terms the rest of the argument is built on
The factory diagram: queue -> harness builds -> automated checks -> review gate -> deploy -> monitoring -> signalsmodel/evidence โ€” identifies the review gate as the single non-scaling, expensive box in an otherwise near-zero-cost pipeline
Dark factory definition, the manufacturing analogy (FANUC, Xiaomi), comprehension debt, and Dex's four-month fully-automated runevidence โ€” a narrative case for the risk of full automation, showing the failure mode is real and discovered late rather than gradually
'The bottleneck was never generation': back pressure defined, the funnel image, and why better models don't close the gapcentral argument โ€” names the constraint and explains why it resists being solved by model improvement alone
Lit factory prescription: judgment moved upstream to design/architecture, and ordinary architectural practice reframed as an agent safety netpractical conclusion โ€” the prescription that follows from the back-pressure argument
'What earns a loop the dark': operational criteria (cheap, frequent, hard-to-fake, immediate, short) for granting full autonomyoperationalization โ€” turns the back-pressure rule into a checkable decision procedure
'Loops, graphs or state machines?': the argument that agentic autonomy is really walking a predefined graph, illustrated with a bug-fix examplesecondary argument โ€” a recommended implementation pattern for making back pressure legible
'Where the human actually goes': the human relocates to owning the outer loop (decide, verify, approve, own consequences)conclusion โ€” resolves the question of what remains for engineers to do

Key terms

Propositions

Arguments

A fully automated ('dark') software factory will eventually fail catastrophically, and the failure will be discovered late rather than gradually. strength: high
Explicit premises:
Implicit premises:
Inferences:
Autonomy should scale with verification capacity, not with generation capacity or model quality โ€” 'back pressure.' strength: medium
Explicit premises:
Implicit premises:
Inferences:
Whether a given loop should run dark or lit is a decidable, criteria-based judgment, not a vibe or a blanket policy. strength: high
Explicit premises:
Implicit premises:
Inferences:
The right way to build reliable agentic systems is to explicitly encode them as directed graphs/state machines with conditional edges, not as an unconstrained loop where the model picks its own path. strength: medium
Explicit premises:
Implicit premises:
Inferences:

Narratives

Questions addressed

Gaps

Closes gaps elsewhere

Related documents