All models, no matter how smart, will eventually build systems...

Steve Yegge · tweet · 2026-09-02 Reader 🖥️

Agentic Engineering Back Pressure (Verification-Bound Autonomy)The Genie (Beck)Scope Discipline Software Design Fundamentals Technical Debt

Any AI coding model, left unconstrained, will eventually build a system that outgrows its own capacity to understand or maintain it — so the durable fix is not a smarter model but an externally imposed cap on system size.

practical · mixed — A general claim (universal, timeless-sounding: 'all models, no matter how smart') bookending a single dated anecdote (Fable 5 outbuilding itself) and a closing practical directive ('keep an iron grip on system size'). Argumentative in structure but grounded entirely in one practitioner's lived experience rather than argument or data.

Structure

Part	Role
General claim: all models, no matter how smart, eventually build systems they can no longer understand or maintain, if allowed to	thesis
Anecdote: Fable 5 'outbuilt itself' and flailed on Yegge for a week; Fable 5.1 looks like it will fix it, for now	evidence
Directive: keep an iron grip on system size, or it will run away from you	prescription

Key terms

outbuilt itself
A coding agent's cumulative output (the system it has built) exceeded its own capacity — within its context/memory/self-model — to track, reason about, or safely modify that system.
iron grip on system size
A deliberate, sustained, human-enforced constraint on how large/complex the agent-built system is allowed to grow, treated as the load-bearing mitigation rather than model quality.

Propositions

explicitAny AI coding model, regardless of capability level, will eventually build a system that exceeds its own ability to understand or maintain, if given unconstrained latitude to keep building.conf: high
extendsMeet the Scope Creep Kraken · p8 — Scope-creep-kraken's 'local productivity can mask systemic degradation of project focus and maintainability' names the same growth-outpaces-comprehension dynamic at the level of a human-AI team's project; Yegge's tweet applies it reflexively to the model itself losing its own grip on what it built.
extendsAI Enthusiasts Are in a Race Against Time, AI Skeptics Are in a Race Against Entropy · p3 — Majors's p3 locates the trust/comprehension breakdown in human engineers unable to read AI-shipped code fast enough; Yegge's tweet extends the same entropy-outrunning-comprehension pattern to the model's own comprehension of its own output, not just the human reviewer's.
explicitFable 5 (Anthropic's model) hit this failure mode directly: it outbuilt itself and produced a week of degraded, flailing behavior for Yegge before recovering.conf: high
explicitA model-side update (Fable 5.1) can substantially mitigate this specific failure mode, at least for now.conf: low
explicitConstraining the size of the system an agent is allowed to build is a necessary, ongoing human responsibility — not a one-time setup choice — because system size will otherwise run away on its own.conf: high
supportsMeet the Scope Creep Kraken · p9 — Both converge on the same mitigation: deliberate, sustained, externally imposed discipline (scope/size constraints) rather than trusting the tool's own momentum or capability.
implicitModel capability improvements (better/smarter models) do not eliminate the need for human-imposed size constraints — they only shift where and when the breakdown occurs.conf: medium

Arguments

You must actively and continuously constrain the size of a system an AI coding agent is building, rather than relying on the model's own judgment or on future model improvements. strength: low

Explicit premises:

All models, however capable, eventually build systems they can no longer understand or maintain if left unconstrained.
Fable 5 did exactly this and flailed for a week as a result.
A model update (5.1) can address a given instance of the problem, but only 'for now.'

Implicit premises:

A model's ability to understand/maintain a system it built is bounded (by context window, working memory, or some analogous limit) rather than unlimited.
System complexity/size tends to grow monotonically over an unsupervised building session unless something actively resists that growth (an entropy-like assumption).
The bound on a model's self-understanding does not scale as fast as its own tendency to keep adding to the system, so given enough unsupervised time the two curves cross.
Human oversight of system size is a cheaper, more durable mitigation than waiting for the model to get smart enough to avoid the problem itself.

Inferences:

From (all models eventually fail this way) + (model updates only patch specific instances, temporarily) infer that the fix cannot be purely model-side, hence must be an externally imposed practice.

Narratives

Yegge used Fable 5 to build (implicitly agentically, over an extended, unsupervised-feeling stretch); the system it produced grew to a point where the model itself could no longer reliably work with it, and Yegge experienced roughly a week of the agent 'flailing' — presumably erratic, unproductive, or self-contradictory behavior — before a model update (5.1) appeared to resolve it.
→ Serves as the tweet's sole piece of evidence, converting the opening universal claim from assertion into (apparent) lived demonstration; also personalizes the prescription that follows, since the 'iron grip' advice reads as something Yegge is now recommending because he didn't do it and paid for it.

Questions addressed

✅ What happens when an AI coding agent is allowed to keep building on a system, across a long stretch of work, without an externally imposed limit on how large or complex that system becomes?
Eventually the system exceeds the model's own capacity to understand or maintain it, producing a period of degraded ('flailing') behavior; the fix that actually holds is a human-enforced cap on system size, not waiting for the model to improve.
⭕ Does a smarter/updated model (5.1 vs. 5) solve this problem at the level of general principle, or only push the same failure further out?

Gaps

incompleteNo mechanism is given for *why* a model loses the ability to understand/maintain what it built — is it a context-window limit, a lack of persistent memory across sessions, an inability to re-derive prior design intent, or something else? Without naming the mechanism, 'iron grip on system size' is a size-based proxy for an underlying limit that isn't specified.
Read next: Read against Anthropic's own Fable 5 prompting guidance (extract-how-to-actually-prompt-fable-5, extract-prompting-claude-fable-5) for its treatment of persistent memory and long-running-agent scaffolding, and against context-engineering material for named mechanisms (context window exhaustion, context rot) that would explain the 'outbuilt itself' failure concretely.
uninformedThe claim rests on a single practitioner's single-tool, single-incident experience (one model family, one week, no quantification of 'flailed'), with no comparison across other frontier coding agents, no data on how large the system actually was, and no independent verification that 5.1 actually fixed the underlying cause rather than coincidentally shipping alongside Yegge changing his own usage pattern.
Read next: Read alongside the corpus's broader over-editing/scope-creep data points (extract-coding-models-are-doing-too-much, extract-scope-creep-kraken) to see whether 'system outgrowing the model's grip' shows up as a measured phenomenon elsewhere, not just as anecdote.
illogicalThe opening claim is stated as a universal law ('all models, no matter how smart, will eventually...') but the very next sentence reports a model update (5.1) apparently fixing the specific instance — which is evidence *against* an unconditional universal, or at minimum requires 'eventually' to be doing a great deal of unexamined work (fixed by 5.1 now just means the runway got longer, not that the law was falsified). The tweet doesn't reconcile this tension; it just moves past it with 'For now.'
Read next: Any future extraction of similar Yegge or practitioner commentary on later Fable versions could test whether the 'runway keeps getting longer but the ceiling remains' framing holds up, or whether at some capability level the failure mode stops recurring altogether.