The Economic Benefit of Refactoring

Giles Edwards-Alexander (published on martinfowler.com, byline Martin Fowler) · rss · 2026-07-30 source ↗ Reader 🖥️

practical · mixed — Primarily a narrated experiment report (narrative + expository data tables), but in service of a practical claim: that refactoring an agent-maintained codebase is worth doing because it measurably lowers future token cost. The author is careful to hedge the practical recommendation with explicit limitations, so it reads as evidence-gathering toward a practical conclusion rather than a confident prescription.

Structure

Key terms

• representative change
  A single fixed prompt describing one small, realistic feature addition (a new trait with three async methods), executed identically by a fresh sub-agent at every measurement point, used as the constant yardstick against which token cost is compared across refactoring stages.
• internal language
  A domain-specific mini-abstraction inside a codebase (e.g. a query-builder DSL for constructing Firestore queries) that a well-factored module should have but this file lacked — its absence is cited as evidence the file was a good refactoring target.
• token cost of change
  The input+output tokens a sub-agent consumes executing the representative change, used as this experiment's operational proxy for a codebase's economic 'friendliness' to agentic modification.
• strict refactoring discipline
  Following Fowler's own definition — 'a provably correctness-preserving series of code edits' — more rigorously than the author believes most human engineers actually do, with each step individually tested.

Propositions

• explicitRefactoring a codebase according to Fowler's refactoring catalog can reduce the token cost of future AI-agent-driven changes to that code.conf: high
  extendsFragments: July 6 · p7 — july-6#p7 asserts token cost as an unvalidated proxy for architectural quality ('if the same change requires fewer tokens, the architecture is better'); this article is the first concrete, named quantitative demonstration of exactly that claim — 83% fewer input tokens after refactoring the same code.
• explicitIn this experiment, the input tokens required for an identical representative change fell from 159,564 (baseline) to 27,360 (after 15 refactoring steps) — an 83% reduction, roughly 132,204 tokens saved per change.conf: high
  extendsFragments: July 6 · p6 — july-6#p6 flags 'whether the ease-of-change lesson carries over to agents' as an open empirical question; this article directly measures ease-of-change as agent token cost, before and after refactoring the identical codebase, and reports a specific number.
• explicitThe token savings occurred because the agent learned to read a smaller, more targeted subset of files, not because there was less code overall — total data-access-layer LoC stayed roughly flat (~16,500-17,155) while it was split into 19 files, and the largest single file shrank from 17,155 to under 3,700 lines.conf: high
• explicitOutput tokens (the code the agent writes) were largely unaffected by refactoring; the savings are concentrated in input tokens (the code the agent must read), and output tokens cost roughly five times as much per token as input tokens.conf: high
• explicitBecause agents don't retain learning across separate sub-agent invocations, a fresh sub-agent can repeat an identical task at each refactoring stage without the improvement-from-practice contamination that would taint the same repeated-measures experiment run by a human engineer.conf: high
• explicitNaively splitting a large file into many smaller files, without first removing duplication, would likely produce smaller token savings than the duplication-first refactoring sequence used here, because the agent would still have to search across many files for the relevant code.conf: medium
• explicitClaude could not identify on its own which refactorings a codebase needed; it required explicit human direction to a specific file and technique, and did not spontaneously flag this file for improvement even though the development harness already included an explicit refactoring step.conf: high
• explicitThe dollar value of this saving is currently small on a per-change basis (39.7 cents at Sonnet 5 pricing for one representative change), and whether it compounds meaningfully across debugging, more complex features, and a whole codebase's worth of refactoring is a question the author raises but explicitly does not answer.conf: high
• implicitcorroboratedToken cost of a fixed representative task can function as an early, measurable proxy for a codebase's 'AI-friendliness' or architectural health — the same underlying idea as 'healthier code yields more reliable AI-generated changes,' but operationalized here as agent read-cost rather than defect risk.conf: medium
  extendsFragments: February 18 · p12 — february-18#p12 operationalizes 'healthy code helps AI' as reduced defect risk in AI-generated changes (Tornhill's research); this article operationalizes the same underlying thesis as reduced agent input-token cost — two different measurable consequences of the same 'codebase health helps agents' idea.
  extends ←Software Design X-Rays · p6 — The hotspot/interest-rate metric — a measured proxy for where real maintenance cost concentrates, mined from version-control history rather than inspected from code alone — is the direct conceptual ancestor of using a measured proxy like agent token cost as an 'AI-friendliness' signal.
  supports ←Code Red: The Business Impact of Code Quality -- A Quantitative Study of 39 Proprietary Production Codebases · p3 — This paper establishes Code Health's predictive validity for human-era defect rates and dev-time variance; the later token-cost experiment extends the same 'code quality as a leading-indicator proxy' logic into agentic token economics.

Arguments

Narratives

• The author watches a live agent edit scroll past line 4,000 of a file in his terminal, looks closer, and discovers the data-access layer has silently grown to 17,155 lines in a single Rust file.
  → Grounds the abstract 'agentic codebases can rot unsupervised' concern in one concrete, surprising moment, and motivates the entire experiment that follows — supports p1 by establishing why the question mattered enough to investigate.
• The full experiment ran mostly unattended for about eight hours on slow hotel WiFi; the only intervention was after 6h40m when it appeared to have finished but had actually silently skipped a step. A bloated cargo build cache, not the WiFi, turned out to be the likely cause of slow test execution.
  → Illustrates both the promise (long autonomous runs are feasible) and the fragility (silent step-skipping needed human catch) of unattended multi-step agentic work, and flags an uncontrolled confound (build-cache state) affecting the reported timing column — supports gap g1/g3 about incomplete experimental controls.
• Using two different Claude interfaces to plan the refactoring, Claude Code identified only 'extract function' as the first step, while Claude.ai went further and spotted an entire client class that should be extracted.
  → A small comparative anecdote showing that interface/context, not just the underlying model, materially shapes an agent's planning quality — supports p7 (Claude needs active human guidance and doesn't spontaneously see the full picture).

Questions addressed

• ✅ Can refactoring a codebase produce a measurable economic benefit for agent-driven software development?
  Yes, in this one case: an 83% reduction in input-token cost for an identical representative change, measured via a controlled repeated-task protocol across 15 refactoring steps.
• ✅ Does refactoring reduce the tokens an agent produces (output), not just what it reads (input)?
  No — the results table shows output tokens stayed roughly flat across refactoring steps; the author attributes this to noise from non-deterministic code generation potentially masking any real effect, and flags it as needing a more complex sample change to investigate further.
• ⭕ Is the token cost of performing the refactoring itself (as opposed to the savings it later produces) known?
  Not precisely — the author admits he didn't think to count it until the experiment was already complete, and can only offer a loose upper bound (~5 million tokens across a broader window that included other mixed work).
• ⭕ Does this result generalize beyond one greenfield, single-developer Rust codebase and one class of change?
  Not established — the author explicitly names this as 'just one experiment' on 'a significant application that is still greenfield and built and maintained by a single developer,' and lists further work (more complex changes, wider refactoring, continuous refactoring, other refactoring approaches) as open.
• ⭕ Is the chars/4 token-count approximation (used because Claude does not expose reliable live token counts) precise enough to trust an 83%-reduction headline figure, or could measurement error itself account for part of the swing?
  Not addressed — the author mentions the approximation once as a caveat and moves on without discussing how much imprecision it could introduce into the reported percentages.

Gaps

• incompleteThe experiment is a single case: one codebase, one change type (adding a trait with three methods), one model/harness (Claude Code / Sonnet 5), one developer, no repeated trials per step to establish variance. This can't yet distinguish 'refactoring reliably reduces agent token cost' from 'this particular refactoring sequence, on this particular file, happened to.'
  Read next: Look for replications across different codebases, languages, and change types. Compare against extract-almedra-token-efficiency-benchmark-glm-5-2, which faces a structurally identical single-task/single-repo generalizability gap from the opposite angle (harness choice rather than codebase refactoring) — the two pieces would benefit from being read together as parallel single-case token-efficiency studies in need of replication.
• incompleteThe token cost of performing the refactoring itself is not measured, only loosely upper-bounded (≤5 million tokens across a broader mixed-work window) — so the piece demonstrates a per-change benefit but not a full return-on-investment (net savings after amortizing the refactoring's own cost against future changes).
  Read next: Look for a follow-up (which the author promises) that precisely counts refactoring-time tokens against the per-change savings to compute a break-even point: how many future changes are needed to 'pay back' the token cost of refactoring.
• incompleteToken counts are estimated via characters ÷ 4 rather than a real tokenizer, because the author states Claude does not provide reliable live token counts. No sensitivity analysis is offered for how much this approximation could shift the headline 83% figure.
  Read next: Look for a follow-up using an actual tokenizer (e.g. re-running the same transcripts through Anthropic's token-counting API) to validate or correct the chars/4 approximation.
• incompleteThe claim that 'randomly cutting the file into smaller files is unlikely to help as much' as the duplication-first sequence used here is asserted, not tested — no control condition (e.g. mechanically splitting the original file into N same-sized chunks without deduplication first) isolates the effect of duplication-removal from the effect of file-count/size on the agent's read behavior.
  Read next: Read extract-fowler-fragments-july-6#g1 and #g2, which flag the identical unresolved confound-isolation problem in the broader 'token cost as architecture proxy' claim this article partially, but not fully, answers.

Closes gaps elsewhere

• Fragments: July 6 · g1 — The gap calls for a methodology isolating architecture from confounds and for empirical studies correlating architectural metrics with agent token cost. This article runs exactly that kind of study: a fixed representative task executed by fresh, memoryless sub-agents at each of 15 refactoring steps on the same codebase and model, isolating code structure as the manipulated variable and reporting an 83% input-token drop. It does not fully resolve the confound question (n=1 codebase/task/model, no per-step variance), but is a direct, named first empirical answer to the gap's own reading direction ('look for empirical studies correlating architectural metrics with agent token cost').
• Fragments: July 6 · g2 — The gap asks for controlled comparisons of agent performance and cost on well-structured vs. poorly-structured codebases. This experiment is precisely that: the same codebase, the same task, and (as far as stated) the same model, with only the code's structure changing across refactoring stages — from a single 17,155-line file to 19 files with the largest under 3,700 lines — measured against the identical representative change at every stage.

Related documents

• Fragments: February 18
• Code maintainability plummets in the AI coding era
  Code Quality Measuring AI Impact Software Design Fundamentals Technical Debt
• Fragments: July 6
  Agentic Engineering Code Quality Software Design Fundamentals Token Efficiency
• [FULL WORKSHOP] AI Coding For Real Engineers - Matt Pocock, AI Hero
  Agentic Engineering Software Design Fundamentals Technical Debt
• All models, no matter how smart, will eventually build systems...
  Agentic Engineering Software Design Fundamentals Technical Debt
• CannotMeasureProductivity
  Code Quality Measuring AI Impact Software Design Fundamentals
• Case Study: Refactoring at Scale with Agents
  Code Quality Technical Debt Token Efficiency
• mattpocock/skills v1.1: does /wayfinder beat grill-with-docs?
  Agentic Engineering Claude Code Code Quality
• Software Design in the Agentic Age: Placing Your Bets
  Agentic Engineering Code Quality Software Design Fundamentals
• You can't unit test for taste
  Agentic Engineering Code Quality Measuring AI Impact