No Vibes Allowed: Solving Hard Problems in Complex Codebases – Dex Horthy, HumanLayer

AI Engineer · video · 2025-12-02 source ↗ Reader 🖥️

practical · argumentative — A conference talk arguing for a specific workflow (research/plan/implement with deliberate context compaction) over both unreviewed 'vibe coding' and an over-literal reading of 'spec-driven development.' Case-study narratives (a successful one-shot fix, a failed migration attempt) function as the evidence base; the talk closes with an explicitly organizational/cultural claim about adoption.

Structure

Key terms

• slop / no slop
  Code (or documentation) that works or looks plausible but that nobody has reviewed, understood, or can vouch for; stale internal docs that have drifted from the real codebase are reframed as 'the amount of lies you can find' — slop in another form.
• smart zone / dumb zone Context Rot & Context Limits
  Borrowed from Jeff Huntley's research: the smart zone is the portion of a context window (roughly up to ~40%, task- and model-dependent) within which the model still reasons and tool-calls well; beyond it, output quality degrades sharply (the dumb zone), and raw context-window size increases ('1M tokens!') do not fix this because they just add more dumb zone.
• mental alignment
  The author's reframing of what code review is actually for: not primarily catching bugs, but keeping everyone on a team synchronized on how and why the codebase is changing. As AI increases code volume beyond what any individual (including a technical lead) can read, mental alignment shifts to happen via review of research/plan artifacts rather than line-by-line diff review.
• compaction / intentional compaction
  Deliberately compressing the working state of a context window into a reviewable artifact (a markdown file with exact files, line numbers, and findings) before continuing or handing off to a fresh agent — contrasted with letting an agent silently drift into the dumb zone or compacting reactively only once things go wrong.
• spec-driven development Spec-Driven Development
  A term the author argues is now broken by semantic diffusion (citing Fowler's 2006 coinage of that pattern, and the earlier fate of the word 'agent'): originally meaning something like 'treat the markdown spec as the source of truth and the code as assembly,' it now means anything from 'a more detailed prompt' to 'a product requirements document' to 'documentation for an open-source library,' making the label useless even though parts of the underlying practice (compressed intent, verifiable feedback loops) remain valuable under other names.

Propositions

• explicitA cited survey of roughly 100,000 developers found that coding-agent use is dominated by rework/churn and performs poorly on complex tasks in large brownfield codebases, while it performs well on small greenfield tasks — matching the author's own experience and conversations with founders and engineers.conf: medium
  supportsPodcast: The AI Joy Gap: Why Some Developers Thrive While Others Struggle · p1 — Both sources independently report the same greenfield-success / brownfield-struggle polarization in developer experience with coding agents.
• explicitdevelopedThe author's three-person team achieved 2–3x coding-agent throughput only after spending eight weeks deliberately redesigning how they collaborated around context management — not from better prompting alone.conf: high
  extends ←Advanced Context Engineering for Coding Agents (ace-fca.md) · p3 — The talk reports the outcome (2-3x throughput after 8 weeks of workflow redesign); the essay names the specific practice that redesign converged on — 'frequent intentional compaction' at 40-60% utilization — and traces the taxonomy of less-sophisticated approaches it supersedes.
• explicitcorroboratedBecause LLMs are stateless (though non-deterministic, not pure functions), the only lever for improving coding-agent output is improving the tokens fed into the context window — the model itself cannot be made smarter mid-task, only its input can be engineered.conf: high
  supports ←12-Factor Agents: Patterns of reliable LLM applications — Dex Horthy, HumanLayer · p6 — Same author's later essay restates and deepens the 'LLMs are stateless, only the tokens you control can be engineered' premise with explicit context-rot data (the 'smart zone'/'dumb zone' finding) that this talk only asserts.
• explicitcorroboratedA conversation history containing repeated correction-and-failure cycles biases the model toward repeating the same failure pattern, because the most statistically likely next token continues the established 'I did something wrong, got corrected, will likely do something wrong again' trajectory.conf: medium
  supports ←LIVE: Uncle Bob on Software Fundamentals in the Age of AI · p3 — Martin's description of an agent breaking one thing while fixing another and going in circles is a concrete instance of Horthy's documented mechanism: a context history full of correction-and-failure cycles biases the model toward continuing that same failure trajectory.
• explicitcorroboratedThere is a 'smart zone' of roughly the first ~40% of a context window (model- and task-dependent) within which output quality holds, and a 'dumb zone' beyond it where quality degrades sharply; staying in the smart zone — via compaction, sub-agent isolation, or restarting — is the central practical lever for getting usable results on hard tasks.conf: high
  supports[FULL WORKSHOP] AI Coding For Real Engineers - Matt Pocock, AI Hero · p19 — Both sources make the same claim in the same vocabulary (smart zone / dumb zone) that larger raw context windows do not expand the usable 'smart' portion — they just add more dumb zone.
  supportsThe Anatomy of an Agent Harness · p5 — Corroborates, from a practitioner framing, the measured finding that context degrades performance substantially once key content sits mid-window ('lost in the middle').
  extends ←Advanced Context Engineering for Coding Agents (ace-fca.md) · p2 — The talk names a single 'smart zone' heuristic (~first 40% of context); the written essay generalizes this into a full four-dimension quality framework (correctness/completeness/size/trajectory) and a ranked failure-mode list, giving the heuristic sharper theoretical structure.
  supports ←LIVE: Uncle Bob on Software Fundamentals in the Age of AI · p8 — Pocock explicitly credits Dex Horthy's 'smart zone / dumb zone' term in this same conversation; this is a direct, named citation of that extraction's proposition rather than an independent convergence.
• explicit⚡ disputed & corroboratedSub-agents' correct use is strictly for context-window isolation and compression (fork a task, return a succinct result to the parent), not for anthropomorphized role specialization (a dedicated 'front-end,' 'back-end,' 'QA,' or 'data-scientist' sub-agent).conf: high
  contradictsZero to Agent in 30 Minutes: From Prompting to Loop Engineering with Ofer Mendelevitch · p4 — That source argues the two-agent coder/reviewer pattern generalizes into larger teams of named-role specialist agents (security, compliance, frontend, backend); this talk explicitly argues against exactly that anthropomorphized-role pattern, saying sub-agents exist for context isolation, not role specialization.
  supports ←A harness for every task: dynamic workflows in Claude Code · p3 — Both independently converge on the same prescription — sub-agents exist strictly to isolate/compress context for a focused goal — from different sources (a practitioner talk vs. the Claude Code team's own product writeup).
  supports ←12-Factor Agents: Patterns of reliable LLM applications — Dex Horthy, HumanLayer · p5 — This talk's 'own your control flow' / sub-agents-only-for-isolation stance anticipates the later essay's explicit rejection of anthropomorphized sub-agent roles in favor of context-isolation-only sub-agent use.
  supports ←Advanced Context Engineering for Coding Agents (ace-fca.md) · p4 — Both make the identical claim (subagents are for context control, not role-play), the essay adding the vivid 'playing house and anthropomorphizing roles' framing explicitly absent from the talk's own wording.
• explicitA three-phase research→plan→implement workflow — research builds an objective, file-and-line-level map of how the system currently works; planning compresses that research plus requirements into an explicit, code-snippet-level plan; implementation executes against the plan while staying in the smart zone — let the author's team ship a one-shot fix validated by an unfamiliar codebase's own CTO, and separately produce ~35,000 lines of code in a single 7-hour session with a colleague, with one resulting PR merged a week later.conf: high
  extendsClaude Code: Best practices for agentic coding · p6 — Both describe essentially the same explore/research → plan → implement workflow; this talk adds the explicit context-budget rationale (staying in the smart zone) and compaction discipline as the reason the sequencing works.
• explicitThe same workflow has real limits: an attempt to remove Hadoop dependencies from a large Java codebase failed outright, and only after abandoning the generated research and plans and returning to first-principles whiteboard understanding did the team identify the actual obstacles ('foot guns').conf: high
• explicit'Spec-driven development' as a term has undergone semantic diffusion (per Fowler's 2006 framing, and the earlier fate of the word 'agent') to the point of uselessness as a label — now meaning anything from a more detailed prompt, to a product requirements document, to literal code-as-assembly, to documentation for an open-source library — even though some version of the underlying discipline (compressed intent, verifiable feedback loops) remains valuable under the author's own preferred framing.conf: high
  extendsHow AI will change software engineering – with Martin Fowler · p9 — Fowler's video (same batch) argues spec-driven development risks the waterfall mistake unless kept thin and tight-looped; this talk goes further, arguing the term itself is now semantically diffused past usefulness, independent of how carefully any given team practices it.
• explicitcontestedThe real function of code review is 'mental alignment' — keeping the whole team synchronized on how and why the codebase is changing — not primarily correctness-checking; as AI-generated code volume grows past what any individual (including a technical lead) can read, review should shift to the plan/research artifact level rather than the code-diff level to preserve that alignment.conf: high
  supportsFrom Async Code Reviews to Co-Creation Patterns · p15 — Both sources converge on the same reframe: the primary purpose of review is building in quality and team alignment/learning, not gating or correctness-auditing in isolation.
  extendsPodcast: The AI Joy Gap: Why Some Developers Thrive While Others Struggle · p4 — That source names the problem (existing PR/code-review tooling isn't designed for AI-generated code volume and needs redesign); this talk offers a concrete redesign — shift review to the plan/research artifact level.
  contradicts ←Bugs in Large Language Models Generated Code: An Empirical Study · p1 — This paper's emphasis on correctness-focused testing to catch specific, named bug categories sits in tension with the claim that code review's real function is 'mental alignment,' not correctness — the taxonomy implies some LLM-specific error categories genuinely need correctness-focused verification that mental-alignment review would not catch.
• explicitStatic, wholesale onboarding documentation for agents should be rejected in favor of progressively sharded, on-demand context (root-level repo context, then sub-directory context pulled in only as needed), because a single large upfront document consumes the smart zone before work starts, and because teams will not reliably sustain the ongoing effort required to keep a static document synchronized with a changing codebase — it will instead drift into stale falsehoods ('slop').conf: high
• explicitPlan design involves a genuine trade-off: longer, more code-snippet-detailed plans increase execution reliability but decrease human readability; teams must find their own sweet spot for their codebase rather than adopt a universal plan format.conf: medium
• explicitdevelopedCoding-agent capability itself is expected to commoditize; the harder, differentiating problem for organizations is adapting team workflow, SDLC, and culture to a world where the large majority of shipped code is AI-authored — and that adaptation must be driven top-down by technical leadership, because it is otherwise producing a growing rift where senior engineers increasingly resent the cleanup burden from AI-generated 'slop' shipped by less experienced colleagues.conf: high
  extends ←Advanced Context Engineering for Coding Agents (ace-fca.md) · p10 — The talk asserts commoditization of agent capability leaves team/culture adaptation as the hard problem; the essay makes this concrete by naming the company's own product response (CodeLayer) and its target customer profile (10-25 person eng orgs), showing what 'driven top-down by technical leadership' looks like as an actual intervention.
• explicitdevelopedAI cannot replace thinking — it can only amplify the thinking a person has already done, or the lack of thinking, if that's what's fed in — so a bad research document or a bad plan compounds into far more downstream bad code than a single bad line would, meaning human review attention should concentrate on research and plans rather than after-the-fact code diffs.conf: high
  supportsHow I Use Agents Without Stopping My Own Growth · p2 — That source diagnoses outsourced thinking as an emergent risk of agent use; this talk independently names the same risk as its central prescriptive warning ('do not outsource the thinking').
  refines ←Advanced Context Engineering for Coding Agents (ace-fca.md) · p12 — The talk's leverage-hierarchy claim (bad research > bad plan > bad code) is largely an assertion; the essay adds the specific mechanism for why it holds in practice — the author's own documented act of reading and discarding a wrong research draft before any plan or code existed, catching the error at the cheapest possible point.

Arguments

Narratives

• The author and the CEO of BoundaryML record a podcast where the author attempts to one-shot a fix to BoundaryML's unfamiliar 300,000-line Rust codebase live; the resulting PR is approved by the company's own CTO, who initially doesn't realize it was done as a demonstration.
  → Shows (rather than merely asserts) that the research/plan/implement discipline can work in a genuinely unfamiliar, large, production brownfield codebase, validated by an independent, initially skeptical party.
• In a follow-up, the author and the same collaborator spend 7 hours on a Saturday shipping ~35,000 lines of code to BAML; one resulting PR is merged a week later, with the collaborator estimating the work would normally take 1–2 weeks.
  → Extends the first story into a claim about not just correctness but velocity on complex work — though the author concedes some of the volume is codegen/golden-file updates, a caveat that qualifies the headline number.
• The author and a different collaborator ('Blake') attempt to remove Hadoop dependencies from a large Java codebase and fail; after burning through generated research and plans, they abandon the AI-assisted approach and return to whiteboard-level first-principles analysis to find the real obstacles.
  → Serves as the talk's own counter-example, establishing the boundary of the method's claimed success and lending credibility to the success stories by showing the author is willing to report failure.
• The film Memento (a man with no memory who must read his own tattoos to reconstruct who he is) is used as an analogy for an un-onboarded coding agent, which will fabricate context if not given it explicitly.
  → Makes vivid, via a well-known reference, why progressive/sharded onboarding context exists and what happens without it — and doubles as the origin of a term ('Memento agent') already used elsewhere in this corpus for the same phenomenon.

Questions addressed

• ✅ Why do coding agents fail at complex tasks in large brownfield codebases, and can this be fixed without waiting for better models?
  Yes, via disciplined context engineering (research/plan/implement plus intentional compaction), demonstrated on a real unfamiliar codebase, with an explicitly named failure case marking the method's current limits.
• ✅ Is 'spec-driven development' a coherent, well-defined practice?
  No — the term has semantically diffused into incoherence, though a version of the underlying discipline survives under the author's preferred framing (research/plan/implement, compaction, verifiable feedback loops).
• ✅ What should sub-agents be used for?
  Context isolation and compression — explicitly not role specialization.
• ⭕ How should teams adapt once the large majority of shipped code is AI-authored?
  Named as 'the hard part' and the central open organizational problem; the only guidance offered is that change must be top-down and cultural, with no concrete governance, incentive, or process redesign specified.
• ⭕ What happens when the agent's actual implementation diverges from the plan that was reviewed for 'mental alignment'?
  Not addressed — the talk assumes the plan is a reliable enough proxy for the resulting code that reviewing it suffices.

Gaps

• incompleteThe talk's two headline successes (the BAML one-shot fix, the 35,000-line 7-hour session) and its one named failure (the Hadoop removal) are all personal anecdotes from the author's own small team and network, delivered by a founder actively promoting his own company's product (HumanLayer, explicitly pitched at the talk's close). There is no systematic success-rate measurement, no control group, and no accounting of how much total effort — including the research and plans he says were 'thrown out' along the way — went into producing the showcased wins, leaving open whether the research/plan/implement discipline reliably generalizes beyond founder-level, highly motivated use.
  Read next: Look for independent, outcome-measured adoption of research/plan/implement-style workflows outside HumanLayer's own network and marketing, ideally with quantified success/failure rates across many tasks.
• incompleteThe talk names a real organizational tension — senior engineers increasingly resenting the cleanup burden from AI-generated 'slop' shipped by less experienced colleagues — and asserts the fix 'needs to come from the top,' but offers no concrete governance model, incentive redesign, or review-process change beyond 'pick one tool and get some reps.' The specific senior/mid-level resentment dynamic it diagnoses is left entirely unaddressed by its own prescription.
  Read next: Look for team-scale-agent-platform or agent-governance treatments that go beyond individual tool adoption and address the specific cross-level resentment/burden-shifting dynamic this talk names but does not solve.
• incompleteThe 'mental alignment via plan review' proposal replaces code review with plan review as the team's synchronization mechanism, but the talk never addresses what happens when the agent's actual tool-calling execution diverges from the reviewed plan mid-task — a known failure mode (agents drifting off-plan, over-editing, or rationalizing scope creep) documented elsewhere in this corpus. Plan review only delivers the claimed alignment benefit if plan and executed code reliably match, which this talk assumes rather than demonstrates.
  Read next: Compare against corpus treatments of agent scope drift and over-editing (e.g. scope-discipline material) to assess how often execution actually tracks an approved plan.

Related documents

• How I Use Agents Without Stopping My Own Growth
• From Async Code Reviews to Co-Creation Patterns
• How AI will change software engineering – with Martin Fowler
• Zero to Agent in 30 Minutes: From Prompting to Loop Engineering with Ofer Mendelevitch
• Podcast: The AI Joy Gap: Why Some Developers Thrive While Others Struggle
• [FULL WORKSHOP] AI Coding For Real Engineers - Matt Pocock, AI Hero
  Context Rot & Context Limits Human-in-the-Loop Multi-Agent Orchestration Progressive Disclosure
• Advanced Context Engineering for Coding Agents (ace-fca.md)
  Agile & XP Context Engineering Context Rot & Context Limits Multi-Agent Orchestration
• The Anatomy of an Agent Harness
  Agent Harness Context Engineering Context Rot & Context Limits Multi-Agent Orchestration
• Harness design for long-running application development
  Agent Harness AI Slop Context Rot & Context Limits Multi-Agent Orchestration
• What is “loop engineering?”
  Agent Harness Context Engineering Context Rot & Context Limits Human-in-the-Loop