The New SDLC With Vibe Coding: From ad-hoc prompting to Agentic Engineering

Addy Osmani, Shubham Saboo, Sokratis Kartakis · pdf · 2026-06-15 Reader 🖥️

Agent Harness Agentic Engineering Code Was Never the Bottleneck Coding Agents Context Engineering Evals Guardrails Human as Orchestrator of Agents Model Routing Software Factory Vibe Coding

As AI increasingly handles code generation itself, what separates production-grade 'agentic engineering' from risky 'vibe coding' is not which AI tool a team uses but how deliberately it designs and maintains the harness, verification (tests + evals), and context around the model — so the discipline a team already has is what AI amplifies, for better or worse.

practical · expository — A practitioner whitepaper (Day-2 of a Google/O'Reilly-adjacent series) that argues teams and orgs should adopt specific practices (context engineering, test+eval verification, deliberate harness design) rather than drift into 'vibe coding'; propositions throughout serve that prescriptive aim even when stated as neutral description of 'how AI transforms each SDLC phase.'

Structure

Part	Role
The shift from syntax to intent: defining AI agents, vibe coding, and the vibe-coding-to-agentic-engineering spectrum (with a comparison table)	context
Context engineering as 'the real skill': six types of context, static vs. dynamic context, Agent Skills as the pattern for managing dynamic context at scale	evidence / mechanism
How AI transforms each phase of the traditional SDLC (requirements, architecture, implementation, testing, review/deployment, maintenance)	evidence
The factory model and Harness Engineering: the developer's output is the system that produces code; 'Agent = Model + Harness'; benchmark evidence that harness changes alone move capability more than model changes	central mechanism / conclusion
The developer's evolving role: conductor (synchronous, in-IDE) vs. orchestrator (async, multi-agent) modes, and the 80% problem	evidence / elaboration
The Economics of AI Development: CapEx/OpEx framing, context engineering as a financial lever, model routing	evidence (economic argument)
Where to start: concrete recommendations for individual developers, engineering leaders, and organizations	conclusion / practical prescription
Conclusion: three durable principles (structure scales, AI amplifies engineering culture, the human role is evolving not diminishing)	conclusion

Key terms

vibe coding Vibe Coding
Karpathy's original sense, narrowed by the authors: prompting an AI, accepting its output without serious verification, and fixing errors by pasting them back into the prompt. Positioned as the low-structure end of a spectrum, not a synonym for 'using AI to code' generally.
agentic engineering Agentic Engineering
The disciplined end of the same spectrum: AI as implementation engine within human-designed constraints, specs, tests, evals, and guardrails, with humans retaining architecture/correctness/quality oversight. Coined (per the paper) by Karpathy himself in early 2026 as a correction to vibe coding's overreach.
harness Agent Harness
Everything around the model that turns a raw model into a working agent: instructions/rule files, tools, sandboxes, orchestration logic, guardrails/hooks, observability. Expressed as 'Agent = Model + Harness.' Team-owned surface area, not the model provider's.
context engineering Context Engineering
The practice of supplying an agent with six structured types of context (instructions, knowledge, memory, examples, tools, guardrails), split into always-loaded 'static' context and on-demand 'dynamic' context (notably Agent Skills), as a discipline distinct from and more important than prompt wording.
the factory model
A mental model where the developer's primary output is not code but the system that produces code: specifications, agents, tests/quality gates, feedback loops, and guardrails. The developer is the factory manager, not the assembler.
conductor / orchestrator
Two modes a developer moves between: 'conductor' = real-time, synchronous, in-IDE, keystroke-level direction of a single AI pair-programmer (inline completion, chat-in-editor); 'orchestrator' = asynchronous, goal-level, multi-agent delegation where the developer reviews outcomes rather than watching code appear (background/terminal agents, CI/CD integration, multi-agent coordination).
the 80% problem
AI agents rapidly generate roughly 80% of a feature's code; the remaining 20% (edge cases, error handling, integration points, subtle correctness requirements) demands deep contextual knowledge current models lack, and increasingly manifests as conceptual rather than syntactic errors that 'look right.'

Propositions

explicitAs of early 2026, 85% of professional developers regularly use AI coding agents, 51% use them daily, and an estimated 41% of all new code is AI-generated.conf: high
explicitcorroboratedThe key differentiator between vibe coding and agentic engineering is not whether a team uses AI, but how much structure, verification, and human judgment surrounds the AI's output.conf: high
supports ←Google Just Dropped a Masterclass on Agentic Engineering (It's SO Good) · p2 — Same spectrum-not-binary thesis from the same source document, restated with the author's own framing that vibe coding is legitimate for POCs/MVPs.
explicitTests verify the deterministic parts of a system; evals verify the non-deterministic parts (trajectory, tool choice, response quality against a bar); without both, the practice is vibe coding regardless of how sophisticated the prompts are.conf: high
explicitThe quality of AI-generated code depends less on prompt cleverness and more on the quality and structure of the context (instructions, knowledge, memory, examples, tools, guardrails) provided to the agent.conf: high
explicitdevelopedAgent Skills let an agent remain a lightweight generalist at session start and flex into specialist roles on demand, so a system can carry many capabilities while only paying the token cost for the one actively in use.conf: high
extends ←Google Just Dropped a Masterclass on Agentic Engineering (It's SO Good) · p5 — Both describe Agent Skills as letting a generalist agent flex into specialist roles; this video adds the explicit argument that this makes complex multi-specialist-agent architectures unnecessary — a claim the sibling extraction doesn't make as directly.
explicitIn agentic engineering, the developer's primary output is the system that produces code — specifications, agents, tests/quality gates, feedback loops, and guardrails — not the code itself (the factory model).conf: high
extendsSoftware Factories, Light and Dark · p1 — Same author's own earlier, more detailed framing of the factory model ('harnessed agentic loops... an org chart made of loops') restated here as the organizing mental model for the whole SDLC paper.
explicitestablishedThe model is only one input into a working agent; the harness (prompts, tools, context policies, hooks, sandboxes, sub-agents, observability) dominates the behaviour developers actually experience, demonstrated by harness-only changes moving a coding agent from outside the Top 30 to the Top 5 on Terminal Bench 2.0, and a separate LangChain study raising a score 13.7 points on the same benchmark by changing only the system prompt, tools, and middleware around a fixed model.conf: high
supportsSoftware Factories, Light and Dark · p10 — Both pin the real constraint on how fast AI-assisted development can run to verification/harness design, not model capability or raw generation speed ('back pressure').
extends ←A harness for every task: dynamic workflows in Claude Code · p2 — new-sdlc frames the harness (not the raw model) as what dominates agent reliability; this article names the specific single-context failure modes (laziness, self-preferential bias, goal drift) that a better harness — here, a dynamic workflow — is built to counter.
extends ←The AI-Native SDLC playbook · p8 — Where the earlier piece names 'the harness' as what dominates agent behaviour without decomposing it, Claxton's playbook operationalizes exactly that dominance into an advisory layer (skills, CLAUDE.md) backed by a deterministic layer (hooks), naming which layer an organization needs for a policy that 'must always hold.'
supports ←Vibe Coding Agent Security And Evaluation · p1 — Both state the same claim in near-identical terms: a model is only one input into a working agent, and the harness (prompts, tools, context, sandboxes) dominates observed behaviour — independent convergence from a security-framework angle and a developer-practice angle.
supports ←Google Just Dropped a Masterclass on Agentic Engineering (It's SO Good) · p1 — Both make the harness-dominates-model claim; this video surfaces the specific 10%/90% split from the same whitepaper's diagram that the sibling extraction described only qualitatively via the Terminal Bench/LangChain benchmark evidence.
supports ←The Anatomy of an Agent Harness · p8 — Both make the same core claim — that the harness, not the model, dominates observed agent behavior and performance — this article's Terminal Bench 2.0 Top-30-to-Top-5 result (model held constant) is a direct empirical instance of that general claim.
explicitcorroboratedMost agent failures, examined honestly, are configuration (harness) failures — a missing tool, a vague rule, an absent guardrail, or a noisy context window — rather than model failures, even though the first instinct is to blame the model.conf: medium
supportsFrom AI Code to Trusted Software: Harness Engineering in Practice · p4 — Both identify verification/harness design, not the model, as where AI-assisted delivery's real bottleneck and real failure surface live.
supports ←Vibe Coding Agent Security And Evaluation · p3 — Restates, almost verbatim, the sibling document's claim that most agent failures trace to a missing tool, a vague rule, or an absent guardrail — i.e. harness configuration gaps rather than model limitations.
explicitcontestedDevelopers move fluidly between two modes: 'conductor' (real-time, synchronous, in-IDE, keystroke-level control, single-file scope, always in the loop) and 'orchestrator' (asynchronous, goal-level, multi-file/multi-agent, reviewing outcomes rather than keystrokes).conf: high
contradictsThe Conductor Developer · p4 — Terminology clash across the corpus: Laycock uses 'conductor' for exactly what this paper calls 'orchestrator' (multi-agent, goal-level delegation, team-lead style); this paper's 'conductor' is the opposite — hands-on, single-IDE, keystroke-level mode. Same word, near-opposite referents.
contradicts ←Google Just Dropped a Masterclass on Agentic Engineering (It's SO Good) · p8 — The source whitepaper (also extracted as extract-new-sdlc-with-vibe-coding#p9) frames conductor/orchestrator as modes a developer fluidly moves between; this video explicitly disagrees, arguing a developer graduates to living in orchestrator mode once the harness is reliable.
explicitcorroboratedThe 80% problem: AI agents can rapidly generate roughly 80% of a feature's code, but the remaining 20% — edge cases, error handling, integration points, subtle correctness — demands deep contextual knowledge current models often lack; AI errors have shifted from simple syntax mistakes to conceptual failures (wrong business-logic assumptions, missed edge cases) that 'look right' and can pass basic tests.conf: high
supportsMeasuring AI impact, assessing readiness, and new data trends · p1 — Both locate the real constraint past code generation: this paper's 80% problem (the hard 20% needs human judgment) and DX's 'code generation was never the real bottleneck' make the same point from different angles.
supports ←Google Just Dropped a Masterclass on Agentic Engineering (It's SO Good) · p9 — Same CapEx/OpEx framing from the same source document, restated with the added '3-10x more reliable and cheaper' crossover figure.
explicitVibe coding has near-zero upfront cost (CapEx) but hides a compounding operational cost (OpEx): token burn from unstructured prompting loops, a 'maintenance tax' from reverse-engineering unstructured AI-generated code, and expensive security remediation after the fact; agentic engineering inverts this profile — high upfront CapEx (specs, tests, structured context), low marginal OpEx.conf: high
explicitContext engineering functions as a financial lever, not just a quality one: a dense, high-signal context payload raises an agent's first-pass success rate and avoids the token-burning trial-and-error loops that plague vibe coding.conf: high
explicitestablishedMature agentic engineering practice routes tasks to different models by cost/complexity — large frontier models for requirements/architecture/complex implementation, smaller cheaper models for deterministic work like test generation, code review, and CI/CD monitoring — to avoid paying premium token prices for trivial tasks.conf: high
supports ←A harness for every task: dynamic workflows in Claude Code · p9 — Both describe routing tasks to different models (Sonnet vs. Opus) by researched complexity/cost rather than a fixed assignment — the same mature-practice pattern named independently.
supports ←How I write software with LLMs · p6 — The author's own architect-is-strong-model/developer-is-cheap-token-efficient-model split is a concrete personal instance of routing tasks to different models by cost and complexity.
explicitcorroboratedIndustry surveys report 25-39% productivity improvements from AI coding tools, but a METR study found experienced developers using AI assistants actually took 19% longer on certain tasks, largely due to time spent verifying, debugging, and correcting AI output — AI transforms implementation work from writing to reviewing/guiding/verifying rather than simply eliminating it.conf: high
supports ←Papers on the UX of AI programming assistants · p6 — Bird et al.'s early-2023 claim that AI integration shifts developer work from doing the task to assessing suggestions anticipates the later, empirically-backed (METR study) finding that AI assistance relocates work from writing to reviewing/verifying.
explicitAI amplifies the engineering culture it lands in: organizations with strong testing practices, clear architectural standards, and healthy code review processes get dramatically more value from AI-assisted development than organizations without those practices.conf: high
explicitThe human role in software engineering is evolving, not diminishing: the bottleneck and the valuable skill shift from implementation to specification, evaluation, architectural judgment, and review, so the most valuable engineers in coming years will be those who can direct agents well rather than those who write the most code.conf: high
supportsThe Conductor Developer · p3 — Both converge on the underlying claim — valuable engineering skill shifts from writing code to directing/evaluating agents at scale — despite disagreeing on which word ('conductor' vs. 'orchestrator') names that directing role.
implicitA team's incentive structure must actually reward the upfront CapEx investment (writing tests/evals before code, maintaining AGENTS.md as reviewed code, owning harness components as shared infrastructure) for agentic engineering's economics to pay off — otherwise the easier, cheaper-looking vibe-coding path wins by default under normal shipping-speed pressure.conf: medium

Arguments

Agentic engineering is the lower-total-cost approach to AI-assisted development at real production scale, despite its higher upfront cost. strength: medium

Explicit premises:

Vibe coding has near-zero CapEx but hides compounding OpEx: token burn, maintenance tax, security remediation.
Agentic engineering has high CapEx (specs, tests, structured context) but low marginal OpEx because output is pretested and structurally sound.
Context engineering raises first-pass success rate, directly cutting the token cost of trial-and-error.

Implicit premises:

The hidden OpEx costs of vibe coding will actually be measured and attributed back to the original coding decision, rather than absorbed invisibly elsewhere in the organization (a different team doing cleanup months later).
The upfront CapEx investment will be recovered within a time horizon the organization cares about, rather than sunk into a system or team that gets reorganized or rewritten before the investment pays off.

Inferences:

Teams that lack cost telemetry or incident postmortems tying bugs/security issues back to code provenance will systematically underinvest in agentic engineering even where it is actually cheaper, because its savings never show up as a line item anyone owns.

Most agent failures, examined honestly, are configuration (harness) failures rather than model failures.

Explicit premises:

Public benchmarks show harness-only changes closing large capability gaps with no model change (Terminal Bench 2.0: outside Top 30 to Top 5; LangChain: +13.7 points via system prompt/tools/middleware alone).
The reflexive instinct when an agent errs is to blame the model, but the failure typically traces to a missing tool, vague rule, absent guardrail, or a noisy context window.

Implicit premises:

Gains measured in one controlled benchmark-competition setting generalize to the much larger, usually unmeasured population of everyday agent failures teams actually experience.
Teams have the observability needed to correctly diagnose a given failure as harness-caused versus model-caused — otherwise the claim 'most failures are configuration failures' is effectively unfalsifiable from inside an ordinary team without that instrumentation.

Narratives

Anthropic's engineering team reportedly had agent teams running on a multi-agent, MCP/A2A-coordinated architecture build a working C compiler in Rust over two weeks in early 2026, with humans setting direction and reviewing output but not writing the implementation.
→ Shows, rather than argues, that orchestrator-mode, multi-agent delegation is already viable for substantial real engineering work — grounding the abstract 'factory model' and 'orchestrator' propositions in a concrete, large-scope example.
Telling a CTO your team is 'vibe coding' their payment processing system will (and should) raise alarm bells; telling that same CTO the team practices 'agentic engineering' — AI implementation under human-designed constraints with test coverage ensuring correctness — is a fundamentally different conversation.
→ Dramatizes why the vibe-coding/agentic-engineering distinction is not academic: it implicitly supports the proposition that the distinction matters most at the point of organizational accountability (what you can defensibly tell a stakeholder), not just at the point of code quality.

Questions addressed

✅ What distinguishes legitimate, production-grade AI-assisted development from risky ad-hoc prompting?
The tests+evals verification criterion, embedded in the broader structure/verification/judgment spectrum.
✅ Why do AI coding agents fail, and what should a team do about it?
Attributed primarily to harness/configuration gaps rather than model limitations, prescribing deliberate harness engineering as the fix.
⭕ Does AI actually make developers faster?
Both a 25-39% productivity-gain figure and a 19%-slower METR finding are cited without reconciling when each applies — the paper leaves 'faster at what, for whom, verified how' open.
⭕ Who inside an organization should bear the upfront CapEx of agentic engineering, and what happens if incentives don't reward that investment?

Gaps

illogicalThe paper cites both a 25-39% industry-survey productivity gain and METR's finding that experienced developers took 19% longer on certain AI-assisted tasks, in the same subsection, without reconciling when each applies — leaving the central 'does this make developers faster' question effectively answered by whichever statistic suits the surrounding argument at that point in the text.
Read next: A study that segments AI productivity gains by task type, developer seniority, or codebase familiarity, rather than reporting a single aggregate figure.
incompleteThe CapEx/OpEx accounting (Figure 9) is a clean two-bucket schematic, but never addresses who inside an organization actually bears the upfront CapEx — often the platform/infra team or a specific engineer, who is rarely rewarded on the same cycle as the feature team that looks fast by vibe coding. The 'for organizations' advice ('treat it as an engineering investment') doesn't engage this incentive-misalignment risk at all.
Read next: Team-topology or incentive-design material on how to attribute/reward harness investment separately from feature-delivery speed.
incompleteThe 'for organizations' section's team/org advice ('plan for hybrid teams of humans and agents... team structures all need to evolve') stops at a bullet point. It never addresses how on-call ownership, incident attribution, or performance review change when an agent authored the code under scrutiny — exactly the durable team-process questions an org the size of a telco would need answered before adopting this at scale.
Read next: Material on agent-authored-code accountability: who is on-call for an agent's code, how incidents get attributed, how review credit/blame is assigned.
closed byThe AI-Native SDLC playbook — Claxton's playbook is the detailed, stage-by-stage answer the gap asks for: it walks through exactly how team/org structure evolves — the product owner reviews intent.md and spec.md, the engineer reviews plan.md and the diff, the tech lead sets REVIEW.md policy and tunes it monthly, the release manager authorizes production deploys via a hook, and on-call shifts from reactive paging to triaging an autonomously-generated intent.md queue in Stage 6 — with governance embedded as hooks and managed settings rather than left as an unaddressed bullet point.
uninformedThe 'most agent failures are configuration failures' claim is drawn from two benchmark anecdotes where harness changes were deliberately isolated and measured in a competition setting; the paper never flags that ordinary teams lack this kind of controlled before/after measurement, so the claim may hold in benchmarks and be unverifiable — not merely under-evidenced, but structurally unfalsifiable — under typical team conditions without comparable instrumentation.
Read next: extract-addy-osmani-quality-in-the-harness.json (same author's companion piece, which itself leaves open who builds/tests/maintains the harness's own checks on an ongoing basis).

Closes gaps elsewhere

Addy Osmani's Post · g1 — That gap asks who builds, tests, and maintains the harness's own constraints on an ongoing basis. This paper's 'for engineering leaders' advice answers directly: treat AGENTS.md, system prompts, eval suites, and skill libraries as code — reviewed in pull requests, versioned with the project, owned by named engineers — framing it explicitly as an ongoing discipline ('without this discipline, the harness drifts') rather than a one-time setup act.