Inside a Software Factory

Paul Iusztin · rss · 2026-09-04 Reader 🖥️

Agent Planning Artifacts Agentic SDLC Context Engineering Human-in-the-Loop Loop Engineering Software Factory Vibe Coding

A 'software factory' — a human-operated automated pipeline of eight SDLC stages over a shared context layer — is the right unit of analysis for disciplined AI-assisted development because it locates humans precisely at brainstorming, planning, and final review while agents execute everything between, and the engineering skill lies not in maximizing automation but in calibrating how much of the factory to build, buy, or leave manual as team scale and trust in the plan grow.

practical · mixed — Primarily expository (defines and maps a taxonomy: eight SDLC stages over a context layer) and argumentative (advances an explicit thesis about where humans belong and when to build vs. buy), with a short narrative section (the author's own overbuilt Squid experiment, and a worked ecommerce example) used to ground the abstractions.

Structure

Part	Role
Opening: against label-chasing, toward 'software factory' as the right frame	Frames the problem
The design of a software factory: eight stages, three buckets, one context layer	Defines the taxonomy
Where the human belongs (worked ecommerce example)	Argues the central thesis via a concrete walkthrough
Don't overbuild the factory (Squid v1 narrative)	Cautionary case study turned design rule
Build vs. buy	Extends the thesis to organizational scale
What's next / closing question	Restates thesis, invites reflection

Key terms

software factory Software Factory
A human-built-and-operated system of eight discrete SDLC stages (triage, brainstorm, plan, implement, review, review-CI, release, monitor) running over a shared context layer, with humans making high-leverage decisions at defined gates rather than doing the execution work themselves.
vibe coding Vibe Coding
Prompting an LLM to produce code without the discipline, review gates, and process (i.e., without a software factory) around it — the thing a software factory is explicitly built to avoid becoming.
loop / graph
A 'loop' is one automatable, invokable step of the factory (e.g., a single /plan or /implement command); a 'graph' is the whole pipeline formed by chaining loops into one end-to-end autonomous command (e.g., /implement-review-all).
context layer
The structured knowledge base (an 'LLM Wiki': analytics, competitor research, docs, transcripts) that feeds brainstorming and planning; its quality directly bounds what those two stages can produce, since with no examples an LLM defaults to the generic solution.
build vs. buy
A threshold, keyed to whether engineers running agents are personally supervised by you, for when to keep building your own factory (skills/agents on a harness) versus adopting an off-the-shelf platform (Factory.ai, Warp's Oz) that absorbs observability/cost/governance overhead — with a further threshold at the largest scale where teams build again because a platform's constraints cost more than replacing it.

Propositions

explicitA software factory is a human-operated pipeline of eight stages (triage/intake, brainstorming, planning, implementing, review, review-CI, release, monitor/incident-response), organized into three buckets (what to build, actual building and checking, self-improving) and run over a shared context layer.conf: high
extendsSoftware Factories, Light and Dark · p1 — Both articles quote Osmani's 'org chart made of loops' framing of a software factory; Iusztin's eight-stage/three-bucket taxonomy is a more granular, SDLC-mapped elaboration of the same underlying model.
explicitThe clear line between vibe coding and disciplined, maintainable use of coding agents is defined by whether a software factory — process and gates — sits around the agent.conf: high
explicitcontestedThe recent wave of '___ engineering' labels (loop engineering, graph engineering) is not wrong so much as it overtheorizes and overexplains practices teams have already been doing since roughly 2024, and risks becoming marketing talk rather than solving delivery problems.conf: high
contradictsLoop Engineering — Iusztin frames 'loop engineering' (and 'graph engineering') as recent labels that 'read more like marketing talk' overtheorizing intuitive practice, in mild tension with Osmani's own loop-engineering piece, which treats loop design as a serious, differentiated engineering discipline superseding prompting skill.
explicitThe quality of the context layer (analytics, competitor research, documentation) disproportionately bounds what the brainstorming and planning stages can produce, because with no relevant examples an LLM defaults to the statistically most common solution rather than the best one for the product.conf: high
explicitHumans are indispensable during brainstorming and planning and return at the final PR review/merge gate; agents own everything in between (implement, QA, review, review-CI, release).conf: high
supportsSoftware Factories, Light and Dark · p12 — Both claim the 'lit'/disciplined factory relocates human judgment upstream into brainstorm/plan (product, design, architecture) rather than only at a final review gate — Iusztin adds the concrete claim that the human also returns at the very end (PR review/merge).
explicitdevelopedTeams should use the strongest, most expensive model for brainstorming and planning and reserve cheaper models for implementation, because a well-written plan lets cheap models execute without reasoning their way out of dead ends, while a weak plan makes them retry until the extra tokens erase the price gap.conf: high
extends ←The Advisor Pattern: Combining Fast Models with Smart Guidance for Scalable AI Systems · p1 — Generalizes the same 'plan with the strong model, execute with the cheap model' practice into a reusable, tool-callable architecture (the advisor pattern) rather than describing a single team's workflow.
explicitTotal agent cost is a function of tokens times price, not model tier, so a cheaper model given a weak plan can out-cost a stronger model on the same task.conf: high
explicitThe software-engineer agent that writes code and the QA agent that checks it must be kept separate, because a model has a positive bias toward its own work and is, in Osmani's phrase, 'too nice grading its own homework.'conf: high
supportsLoop Engineering · p7 — Identical claim (agents are biased toward grading their own work favorably, so implementer and QA/reviewer must be separate agents), attributed to the same source (Addy Osmani) in both pieces; also echoed in extract-agent-harness-engineering#p7.
explicitWhen an implement/QA feedback loop keeps failing, the root cause is almost always missing plumbing (e.g., no single reproducible command to start the whole stack) rather than a limitation of the agents themselves.conf: medium
explicitThe author's first, fully autonomous Squid pipeline (big remote workflows, parallel agents, one end-to-end run) became undebuggable and uncontrollable the moment something went off-script, because it offered no way to halt, inspect, or redirect a run without discarding it entirely.conf: high
supportsSoftware Factories, Light and Dark · p9 — Both report a real personal failure of a fully autonomous, human-out-of-the-loop pipeline that went undetected/uncontrollable for a period before being caught — Osmani's account is more concrete (about four months, a painstaking manual debug) and can be read as corroborating detail for Iusztin's vaguer 'something went offscript' story.
explicitA software factory should expose two interfaces — granular single-step commands for human-directed control, and one chained end-to-end command for full autonomy — with planning kept as its own separate, always human-driven command regardless of which mode is used.conf: high
explicitThe right amount of factory automation scales with team size and should be earned incrementally — start by automating the most time-consuming bottleneck and add complexity only as people get comfortable — rather than defaulting to maximal automation.conf: high
explicitFeature-level parallelism is inherently limited for most projects, because most features build on one another; the number of independent, safely parallelizable features grows only slowly as a project grows, which is why the author only ever parallelizes with local, worktree-isolated agents rather than fleets of 24/7 remote agents.conf: medium
explicitTeams should build their own factory (skills/agents on a purchased coding harness) while small, and cross to buying an off-the-shelf platform (Factory.ai, Warp's Oz) once engineers who are not personally supervised begin running agents, because observability, tracing, cost tracking, and billing then stop being optional.conf: high
extendsFrom AI Code to Trusted Software: Harness Engineering in Practice · p8 — Both propose a staged progression of where human oversight sits as agentic delivery matures (the harness-engineering piece's four-stage Assisted→Harnessed→Risk-tiered→Autonomous maturity arc); Iusztin's version keys the threshold specifically to whether engineers running agents are personally supervised, and adds a build-buy-build curve on top.
explicitAt the largest scale, organizations cross back from buying to building once a platform's constraints cost more than the team it would take to replace it — illustrated by OpenAI's internally built, Codex-based product (about 1M lines and about 1,500 merged PRs over five months with zero hand-written lines).conf: medium
explicitcontestedA software factory is fundamentally about process, not tools; a factory that doesn't fit how a team already works adds friction, never gets adopted, and ends up useless.conf: high
contradicts ←Cloudflare Introduces the Agent Development Lifecycle to Replace Traditional SDLC · p8 — extract-inside-a-software-factory#p16 argues a software factory is fundamentally about process, not tools, and that a factory which doesn't fit how a team already works never gets adopted; this article pitches the opposite — adopt Cloudflare's specific product stack as the prerequisite infrastructure for the ADLC.
explicitEven in a future where almost the entire SDLC is automated, brainstorming and planning will remain the exceptions — the stages that keep requiring the human the most.conf: high
supportsThe keyboard was never the bottleneck · p7 — Both converge on the same claim: as AI absorbs more of the execution work, the durable human value shifts toward planning, architecture, and judgment about what's worth building — Iusztin frames it as the factory's permanent bottleneck, Amundsen as levels 3-4 of a four-level taxonomy.
explicitA claim that someone has 'cracked' the software factory problem should be read skeptically: it usually indicates either insufficient testing of the idea or an attempt to sell it.conf: medium

Arguments

Humans should own brainstorming, planning, and the final PR review; agents should own everything between (implement, QA, review, review-CI, release). strength: medium

Explicit premises:

In brainstorming, agents can do the grunt work (scanning competitors, pulling research) but a human interprets why users disengage and proposes the fix as a business-level feature spec.
In planning, a human works out feasibility, architecture, interfaces, data flow, cost, and latency, then lets the agent grill them until the plan fits the codebase.
OpenAI's Codex-built product shipped ~1M lines and ~1,500 merged PRs over five months with zero hand-written lines, evidencing that agents can reliably own the implement-through-release span once given a plan.
The 'factory' still ends with a PR that a human must review and merge.

Implicit premises:

Business/product taste and judgment about what's worth building are not capabilities current coding agents reliably possess.
The quality of everything downstream (implementation, review, release) depends critically on the plan being correct, so cheap execution cannot compensate for a bad plan.
A single well-documented extreme case (OpenAI) generalizes to a broader claim about where agents can safely be trusted with full ownership.

Inferences:

B
e
c
a
u
s
e
a
g
e
n
t
s
l
a
c
k
r
e
l
i
a
b
l
e
t
a
s
t
e
/
j
u
d
g
m
e
n
t
b
u
t
a
r
e
c
h
e
a
p
a
n
d
e
f
f
e
c
t
i
v
e
o
n
c
e
a
t
a
s
k
i
s
w
e
l
l
-
s
p
e
c
i
f
i
e
d
,
t
h
e
o
p
t
i
m
a
l
d
i
v
i
s
i
o
n
o
f
l
a
b
o
r
p
l
a
c
e
s
h
u
m
a
n
s
a
t
t
h
e
j
u
d
g
m
e
n
t
-
h
e
a
v
y
,
l
o
w
-
v
o
l
u
m
e
s
t
a
g
e
s
(
b
r
a
i
n
s
t
o
r
m
,
p
l
a
n
,
f
i
n
a
l
r
e
v
i
e
w
)
a
n
d
a
g
e
n
t
s
a
t
t
h
e
w
e
l
l
-
s
p
e
c
i
f
i
e
d
,
h
i
g
h
-
v
o
l
u
m
e
s
t
a
g
e
s
(
i
m
p
l
e
m
e
n
t
t
h
r
o
u
g
h
r
e
l
e
a
s
e
)
.

A software factory should never be built as a single, maximally autonomous end-to-end pipeline with no ability to halt or redirect mid-run; it should expose granular, human-directed commands as well. strength: low

Explicit premises:

The author's first Squid version chased full autonomy (big remote workflows, parallel agents, one grand end-to-end pipeline).
It worked until something went off-script, which the author says 'usually' happens.
When it went off-script, the author could not debug it, halt it mid-run, or redirect it without discarding the entire run.

Implicit premises:

Runs going off-script is common enough across factories in general (not just this one implementation) that lacking a halt/redirect mechanism is an unacceptable design risk, not a one-off bug to fix.
A single personal case study is sufficient grounds for a general architectural rule about how any software factory should be structured.

Inferences:

F
r
o
m
o
n
e
c
o
n
c
r
e
t
e
f
a
i
l
u
r
e
o
f
a
s
p
e
c
i
f
i
c
f
u
l
l
y
-
a
u
t
o
n
o
m
o
u
s
i
m
p
l
e
m
e
n
t
a
t
i
o
n
,
t
h
e
a
u
t
h
o
r
i
n
f
e
r
s
a
g
e
n
e
r
a
l
p
r
e
s
c
r
i
p
t
i
o
n
(
o
f
f
e
r
b
o
t
h
g
r
a
n
u
l
a
r
c
o
m
m
a
n
d
s
a
n
d
a
n
o
p
t
i
o
n
a
l
c
h
a
i
n
e
d
-
a
u
t
o
n
o
m
y
c
o
m
m
a
n
d
)
f
o
r
a
l
l
s
o
f
t
w
a
r
e
f
a
c
t
o
r
i
e
s
.

Teams should build their own software factory while small, buy an off-the-shelf platform in the middle regime (once unsupervised engineers run agents), and build again at the largest scale. strength: medium

Explicit premises:

Once engineers you don't personally supervise run agents, observability, tracing, cost tracking, and billing stop being optional and become a full-time job.
Off-the-shelf platforms (Factory.ai with Droid, Warp's Oz) exist specifically to absorb that governance/observability burden.
OpenAI's internally-built, Codex-based pipeline is cited as evidence that at the largest scale, a platform's constraints can cost more than the team needed to replace it.

Implicit premises:

Governance and observability needs are similar enough across companies in the 'middle' regime that a generic vendor platform can substitute for bespoke in-house tooling there.
The point at which platform constraints exceed replacement cost is itself scale-dependent in a predictable, monotonic way (a smooth build→buy→build curve), rather than being idiosyncratic to each organization's specifics.

Inferences:

A
U
-
s
h
a
p
e
d
b
u
i
l
d
/
b
u
y
c
u
r
v
e
a
c
r
o
s
s
o
r
g
a
n
i
z
a
t
i
o
n
a
l
s
c
a
l
e
,
a
n
c
h
o
r
e
d
b
y
t
w
o
e
x
t
r
e
m
e
,
n
o
n
-
a
d
j
a
c
e
n
t
d
a
t
a
p
o
i
n
t
s
(
t
h
e
a
u
t
h
o
r
'
s
o
w
n
s
o
l
o
p
r
a
c
t
i
c
e
;
O
p
e
n
A
I
'
s
s
e
l
f
-
r
e
p
o
r
t
e
d
c
a
s
e
)
.

Narratives

The author's own first version of Squid (his software factory) chased full autonomy — big remote workflows, parallel agents, one grand end-to-end pipeline — worked until something went off-script, and then could not be debugged, halted, or redirected without discarding the run; he says he was 'so overbuilt' he stopped using it.
→ Cautionary case study that grounds the 'don't overbuild the factory' section and motivates the granular-commands-plus-optional-autonomy design rule.
A worked hypothetical: a shopping-assistant recommendation feature on an Amazon-like ecommerce platform, walked stage by stage from brainstorm (a human interprets usage data and competitor research to propose a spec) through plan (a human works out architecture/cost/latency with agent help, producing tickets and an ADR) through implement/QA/review to a mergeable PR.
→ Makes the abstract eight-stage taxonomy concrete and demonstrates exactly where the human is 'central' versus where agents 'own' the work.
OpenAI's own reported case of a Codex-built product: roughly 1M lines of code and about 1,500 merged PRs shipped over five months with zero hand-written lines, framed by OpenAI as 'Humans steer. Agents execute.'
→ Cited as extreme-case evidence both for 'agents own everything between planning and final review' and, later, as the anchor example for crossing back from buying to building at the largest scale.

Questions addressed

✅ Where does the human belong in an AI-driven software delivery pipeline?
At brainstorming and planning (where taste and business judgment live), and again at the final PR review/merge gate; agents own implement, QA, review, review-CI, and release in between.
✅ What's worth automating, and how much automation should a given team build?
Automate incrementally, starting from the biggest time bottleneck, scaling automation ambition with team size and comfort rather than defaulting to maximal end-to-end autonomy.
✅ When should a team buy an off-the-shelf software-factory platform instead of building its own?
Once engineers you don't personally supervise start running agents, because governance/observability/cost-tracking needs then require dedicated tooling; teams cross back to building only at the largest scale, when a platform's own constraints cost more than replacing it.
✅ How should a software factory be architected so it stays controllable rather than becoming an unhaltable black box?
Expose granular, single-step commands for human-directed work, plus a separate optional chained command for full end-to-end autonomy — keeping planning as its own permanently human-driven command regardless of mode.
⭕ Exactly how many features (or what proportion of a codebase) can safely be parallelized as a project or team grows?
⭕ What measurable criteria distinguish a plan good enough to yield a near-ship-ready PR from one that isn't?

Gaps

incompleteThe claim that a strong plan makes the resulting PR 'usually ready to ship as-is' is asserted without any measurement — no defect rate, revision count, or review-time comparison against PRs produced with weaker plans is given, so a reader cannot tell how much of the plan-quality effect is real versus optimistic self-report.
Read next: Read against extract-addy-osmani-software-factories-light-dark, whose back-pressure argument (only automate what you can cheaply and reliably verify) implies a measurable verification standard this article never applies to its own central claim; also compare extract-agent-cost-attribution's push for turn-level instrumentation as the kind of evidence that would substantiate or falsify the claim.
incompleteThe Squid v1 failure ('something went off-script... I couldn't debug it, couldn't halt it mid-run') is never described concretely — what broke, how it manifested, how long it ran undetected — so it functions as an origin myth justifying the granular-vs-autonomous design rule without evidence that granular commands would actually have prevented the same failure on a different task.
Read next: Compare against extract-addy-osmani-software-factories-light-dark#p9, which reports a similar personal failure (a fully automated pipeline running about four months before a serious bug was found) in far more concrete terms — useful for testing whether Iusztin's vaguer account describes the same underlying failure mode.
incompleteThe build→buy→build-again curve across organizational scale rests on exactly two data points at the extremes — the author's own solo practice at the small end, and OpenAI's self-reported internal Codex pipeline at the largest end — with no examples, metrics, or switching-cost discussion from the broad middle range (roughly 10-100 engineers) where most organizations actually operate.
Read next: Look for practitioner reports from mid-size engineering orgs that have adopted or abandoned Factory.ai or Warp's Oz, to test whether the two-endpoint curve generalizes or whether the middle regime behaves differently than either anchor suggests.
incompleteThe claim that brainstorming and planning 'will remain' the human-heaviest stages 'even in a world where all the code is generated by AI' is stated as a durable structural fact, but no mechanism is given for why taste and business judgment specifically resist automation, as opposed to simply being what current-generation agents happen not to do well yet.
Read next: Read against extract-the-keyboard-was-never-the-bottleneck's four-level taxonomy of programming work and its own acknowledged gap (whether AI's weakness at higher-level judgment is a durable limitation or a current snapshot) to test whether 'planning stays human forever' is a reasoned claim or an extrapolation from present capability limits.
illogicalThe article opens by criticizing '___ engineering' neologisms (loop engineering, graph engineering) as marketing talk that overtheorizes intuitive, already-established practice, then spends its own middle section formalizing 'software factory' into an eight-stage taxonomy with its own new vocabulary (context layer, ADR-plus-glossary as planning output, grill-the-plan) — the piece never explains why 'software factory' escapes the exact critique it levels at sibling terms.
Read next: Read against extract-osmani-loop-engineering, which shows the identical pattern (a piece critiquing terminology churn while itself coining new terminology) — useful for testing whether this is a recurring rhetorical feature of the 'next ___ engineering' genre rather than something specific to this author.

Inside a Software Factory

Structure

Key terms

Propositions

Arguments

Narratives

Questions addressed

Gaps

Related documents