Brownfield Agentic Engineering
Addy Osmani · article · 2026-09-14 source ↗ Reader 🖥️
Agent HarnessAgent Work LedgerBack Pressure (Verification-Bound Autonomy)Legacy & Organizational ForgettingPull Requests & Async Code ReviewRisk TieringVerifiability (as the Quality Bar)
Agentic engineering in an old (brownfield) codebase succeeds by making hidden, undocumented constraints explicit and the cost of a wrong change cheap and visible — through human-drawn risk zones, durable research artifacts, characterization tests, migrations completed as whole units, and review that scales with blast radius rather than code volume — not by maximizing agent autonomy or parallelism first.
practical · mixed — A practitioner playbook (slide-deck-derived) for operating agents in existing codebases, built from prescriptive patterns, cited case studies/benchmarks, and two personal anecdotes.
Structure
| Part | Role |
|---|---|
| Brownfield redefined + 'write down what the code can't say' | framing / premise |
| Zones (green/yellow/red) and the three rules that make them a procedure | core mechanism |
| Durable research artifacts and the harness ('every repeated correction is a missing piece of the harness') | core mechanism |
| Zero-risk-first sequencing: characterization tests, mechanical transforms, shadow-traffic diffing | evidence / method |
| Migrate in complete units + the AOL homepage anecdote | evidence / method |
| 'Receipts' from bigger migrations (Bun, VB6-to-C#, Stripe, Spotify, Asana) | comparative evidence |
| Parallelize last + automated review design | conclusion / prescription |
Key terms
- brownfield Legacy & Organizational Forgetting
Any codebase an agent writes into without decision-by-decision human authorship — not only codebases that are literally old. The defining property is that the repository is no longer a complete description of actual behavior. - zone (green/yellow/red)
A human-assigned risk classification of a codebase region that mechanically sets the required verb for agent work there (tight loop / tests-first / pair-or-don't) — a procedure, not a vibe. - characterization test
A test that pins down what a module does *today*, ugly or wrong behavior included, specifically so that an agent (or any refactor) doesn't 'fix' behavior the business actually depends on behind a green suite. - the harness (as record of corrections)
The specific claim that every repeated human correction of an agent is a diagnosable gap in the harness, and that the fix is to promote it up a ladder — quietly fixed diff, review comment, written instruction, skill, lint/hook/type/test, deny rule/CI/scoped credential — until the correction no longer needs a person to remember it. - complete migration unit
One route migrated fully end-to-end, with the new path serving it and the old dependency demonstrably, verifiably gone — not partial conversion with deletion deferred to a future ticket.
Propositions
- explicitBrownfield status is a property of any codebase an agent writes into without decision-by-decision human authorship, not only codebases that are literally old.conf: highsupportsCase Study: Refactoring at Scale with Agents · p1 — Both treat poor codebase structure/technical debt as a direct, measurable constraint on agent performance, not just a human-maintainability concern — motivating the zone/characterization-test discipline here and the Code-Health-first discipline there.
- explicitBecause agents can already infer most of a codebase's structure from the code itself, written context should be reserved for exactly what the code can't say — business/team nuance, trade-offs, unenforced conventions, domain rules, and history.conf: high
- explicitCodebase risk zones (green/yellow/red) must be drawn by a human, not inferred by the agent, because an unconstrained agent gravitates toward the scariest-looking file rather than the actually-sensitive one.conf: high
- explicitA zone should only be promoted (e.g. yellow to green) once it's earned — specifically, once characterization tests exist for it and its owner has reviewed the agent's first changes there.conf: high
- explicitWithout a durable, source-cited comprehension memo produced by a read-only research pass, each new agent session re-pays the cost of the same codebase archaeology, because chat history is not a reliable system of record, especially after context compaction.conf: high
- explicitA repeated correction that is only fixed in the diff, rather than promoted into a lint rule, hook, type, test, skill, or deny/CI rule, will recur, because relying on someone remembering it doesn't scale.conf: highsupportsMaintainability sensors for coding agents · p7 — Both treat the record of an agent's exceptions/corrections (suppressed warnings, repeated review comments) as a more efficient and higher-signal review entry point than reading the whole diff.
- explicitAn agent should not both author the characterization tests that pin current behavior and make them pass in the same session, because that collapses into a green suite that merely encodes whatever the agent itself just implemented.conf: high
- explicitWhen no honest, trustworthy test suite exists for a surface (e.g. a long-lived homepage with partial coverage), the substitute for unit tests is running old and new implementations side by side against real or replayed traffic and diffing the outputs, promoting only on a match.conf: highsupportsCase Study: Refactoring at Scale with Agents · p3 — Both use a shadow/replay-comparison oracle (diff old vs. new behavior on real traffic or recorded state) as the substitute for a trustworthy test suite — the same pattern applied to a web backend here and to a game-engine refactor there.
- explicitcontestedA migration unit is complete only when the new path serves the route and the old dependency is demonstrably, verifiably removed — not when many files have been converted while a shim keeps both patterns alive — because half-finished migrations present agents with contradictory precedent and tests can stay green while a replacement silently still calls legacy code ('migration blindness': only 28 of 520 agent runs passed full audit per SWE Refactor Bench).conf: highcontradictsThe Hard-Stop Rule: From 3 HCM Monoliths to 120 Domain Microservices · p15 — Osmani requires every migration unit to end with the old path demonstrably gone; the pull-based HCM migration explicitly accepts that 'cold corners' nobody touches stay permanently unmigrated. Both are coherent strategies, but they disagree on whether a permanently-incomplete migration state is acceptable.
- explicitWhat transfers between companies' large-scale agentic migrations is the surrounding structure (porting guides, adversarial reviewers, pre-existing test suites as merge gates, disposable trial runs) — not the headline cost/time figures, several of which are vendor-reported rather than controlled comparisons.conf: high
- explicitParallelizing agent work should come last, because parallelism multiplies whatever review bottleneck already exists; a single unit of work needs a dependable automated judge, a cheap recovery path, and an absorbable review format before it is replicated many times over.conf: highextendsHuman judgment doesn't leave the software factory. It relocates. · p7 — Both make the same point from the same author: human cognitive bandwidth doesn't scale with parallel agent sessions, so parallelizing before a unit has a dependable judge just multiplies an existing review bottleneck.
- explicitAutomated review should present intent, changed invariants, test results, parity mismatches, and the rollback route first, with the full diff available but secondary, so human attention goes first to the largest blast radius and weakest oracle.conf: high
- explicitGit worktrees isolate changes, not behavior: they can still share credentials, local services, and network access, so unattended agents working on untrusted content need real sandboxing, not worktree separation alone.conf: high
- explicitAgents make the longstanding cost of tribal, undocumented conventions newly visible and countable (as recurring review comments, interventions, rollbacks) rather than creating that cost — strengthening, not inventing, the case for maintenance work teams already knew was valuable.conf: high
- implicitAutonomy for a given change should be set by that change's blast radius, observability, and recoverability — not by the model's apparent confidence, and not as a single project-wide setting.conf: highextendsHuman judgment doesn't leave the software factory. It relocates. · p3 — Same author's companion piece frames checks as a sequenced 'verification budget' by cost and value; this article's zone/blast-radius framing is the same idea applied specifically to how much autonomy a change earns in a brownfield codebase.
Arguments
The rigor and autonomy applied to an agent's work should be set by the risk zone of the code it's touching, not by the agent's or model's general capability or confidence. strength: high
Explicit premises:
- Zones are defined by test coverage, convention currency, and isolation (green), mixed quality (yellow), or sensitivity (red).
- Each zone sets its own verb — tight loop, tests-first, or pair-or-don't.
- Zones are promoted only when earned (characterization tests plus owner review), never pre-emptively.
Implicit premises:
- A model's stated or apparent confidence is not a reliable signal of actual risk.
- The organization — not a generic tool setting — must own the risk classification of its own codebase.
Inferences:
- The same codebase can legitimately be governed completely differently (full autonomy vs. none) depending on location, which argues against blanket, project-wide 'autonomy level' settings for an agent or tool.
Migrations should be executed and verified as complete, narrow units (one full end-to-end route, old path deleted) rather than broad, partial conversions, even though broad partial conversion looks faster to start. strength: medium
Explicit premises:
- Half-finished migrations create contradictory precedent (old approach in 40 files, replacement in 12, a shim presenting both as current) that confuses agents.
- Tests can stay green while a replacement silently still calls legacy code ('migration blindness'); only 28 of 520 agent runs in SWE Refactor Bench passed full audit.
- 'If deletion is a future cleanup ticket, the migration unit is not complete.'
Implicit premises:
- Cleanup tickets to remove an old path are systematically deprioritized once the new path already works.
- A codebase left in a mixed, half-migrated state is more dangerous to work in with agents than one that is either fully old or fully migrated.
Inferences:
- Scoping migrations this narrowly trades calendar breadth (fewer files touched per unit of effort) for correctness confidence, implying migration schedules should be planned in units-of-completion rather than files-converted.
Narratives
- Called in on a day off to fix the completely broken AOL.com homepage — a page owned piecemeal by dozens of departments' components, A/B tests, and scripts, with uneven unit-test coverage — and having to user-test the uncovered parts to confirm nothing else broke.
→ Grounds the 'dozens-of-departments problem' and the claim that a surface only production traffic really understands is a red zone by definition; establishes that user-testing-as-gate for under-tested surfaces predates agents entirely, and that agents don't remove this problem, only make attempts against it cheaper. - GitHub's Scientist library and Netflix's GraphQL cutover both ran old and new code paths side by side against real/replayed traffic, returned the trusted old answer to users, and diffed the new answer in the background, promoting only on a match.
→ Narrative-as-evidence for the shadow-traffic-diffing pattern — shows it as an established production technique being repurposed for the agent era, not invented for it, which lends the prescription external credibility. - A comparative roundup of large migrations — Bun's Zig-to-Rust port (adversarial reviewers, full suite as merge gate, porting guide), a controlled VB6-to-C# study (92% vs. 47% behavioral equivalence by complexity), Stripe's pre-agent 3.7M-line TypeScript migration via codemods, Spotify's 650+ monthly agent PRs on pre-built rails, and Asana's vendor-reported $12k Enzyme migration.
→ The common structure across very different cases (narrow mechanical scope, pre-existing test suite, humans still reviewing) is the actual argument — 'copy the structure, not the headline' — turning five disparate anecdotes into one comparative claim about what's transferable.
Questions addressed
- ✅ How much autonomy should an agent get in a given part of a brownfield codebase?
Set by the code's risk zone — a function of blast radius, observability, and recoverability for that specific change — not by the agent's or model's general confidence. - ✅ When is a migration actually finished?
When the new path works and the old dependency is demonstrably, verifiably removed — not when most files have been converted. - ⭕ Under a 'complete units only' migration discipline, what happens to code that never receives a feature or fix request and so never becomes a migration unit?
Gaps
- incompleteThe piece prescribes migrating in complete units with the old path fully deleted, but never addresses what happens to code that simply never receives a feature or fix request — the 'cold corners' that a pull-based/strangler-style migration explicitly concedes stay unmigrated. Without some forcing mechanism, complete-unit migration may structurally never reach large parts of a sprawling brownfield system.
Read next: Compare against pull-based/strangler-fig migration models and how they handle (or accept) permanently-unmigrated cold corners. - incompleteThe 'receipts' section is careful to flag that other companies' headline migration numbers are vendor-reported or otherwise uncontrolled — yet the article's own prescriptions (zones, characterization-test-first, complete units, parallelize-last) are offered without any comparably rigorous measurement of whether following them actually produces better outcomes than not following them. The piece asks for more rigor from others' numbers than it applies to its own recommendations.
Read next: Look for controlled or telemetry-based studies (e.g. DORA, Faros) that test whether zone-based governance and complete-unit migration actually reduce defect/incident rates versus less structured approaches. - uninformedThe zone model assumes a human owner exists who can draw the map and review an agent's first changes in a yellow or red zone — but doesn't address the case where the people who understood the red-zone code (billing, payroll, auth) have already left, which is exactly the organizational-forgetting scenario brownfield systems are prone to.
Read next: Look at organizational-forgetting / corporate-memory literature for how teams reconstruct ownership and tacit knowledge for sensitive code once the original owners are gone.
Closes gaps elsewhere
- Presentation: Accelerating Performance by Incrementally Integrating Rust Into Existing Codebase · g2 — Supplies the general stopping rule that talk was criticized for lacking: a migration unit converges when the new path works and the old dependency is demonstrably, verifiably removed — giving incremental migration a defined endpoint per unit, analogous to a strangler-fig migration's natural endpoint, rather than an indefinite process.
Related documents
- Maintainability sensors for coding agents
- Case Study: Refactoring at Scale with Agents
- The Hard-Stop Rule: From 3 HCM Monoliths to 120 Domain Microservices
- Software Factories, Light and Dark
- Loop Engineering: Building Blocks, Adoption, and Impact
- How building software is changing at Anthropic
- Article: When Spec-Driven Development Pays Off
- Addy Osmani's Post
- Podcast: The AI Joy Gap: Why Some Developers Thrive While Others Struggle
- AWS Expands DevOps Agent with AI-Powered Release Management to Validate Code Before Production