Agentic AI and Code Reviews

Zach Davis, Michelle Gill, Elisabeth Hendrickson, Angie Jones, Sha Ma, Randy Shoup, James Wickett · pdf · 2026-09-14 source ↗ Reader 🖥️

Agent Security Coding Agents Context Engineering Feedback Loops Multi-Agent Orchestration Pull Requests & Async Code Review Spec-Driven Development Trust

As AI agents generate code faster than human attention can scale to review it, code review must be reorganized from a single human merge-time gate into a layered, risk-tiered verification system — deterministic checks, continuous author/review-agent loops, structurally independent adversarial review, and specialist multi-agent review — with human judgment deliberately reserved for intent, strategic risk, and accountability rather than mechanical inspection, and every automated decision recorded as a durable, auditable artifact.

practical · mixed — A named pattern language (explicitly modeled on Christopher Alexander) argued into existence through empirical motivation (review-capacity bottleneck), illustrated with three detailed organizational case studies (Block, Cloudflare, DryRun Security), and closed with direct recommendations to engineering leaders — practical throughout, with expository pattern definitions and narrative case evidence.

Structure

Part	Role
Introduction and Defining Code Review — establishes the capacity crisis and a precise three-part definition of code review	context / framing
The Value of Code Reviews — cites empirical studies showing review's value extends well beyond defect-finding to maintainability and knowledge sharing	evidence / stakes
How Does Agentic Code Review Work — reframes trust as earned, not assumed, and lays out the generation→verification→release→learn loop with risk-tiered human involvement	thesis statement
Patterns for Agentic Code Review (nine named, composable patterns: Small Incremental Changes, Shift Left, Design for Verification, Deterministic Engineering Practices, Context Engineering, Continuous Code Quality Loop, Adversarial LLM Review, Multi-Agent Reviews, Spec-Driven Development)	core content / toolbox
Case Studies (Block, Cloudflare, DryRun Security) — show three independently-arrived-at versions of the same layered system under real organizational constraints	evidence / validation
Concluding Thoughts — reframes the shift as organizational redesign (responsibility, risk allocation, role blurring, audit records) rather than a tooling upgrade	conclusion

Key terms

Code review Pull Requests & Async Code Review
An examination of source code by an entity that is not the author, with the intention of making a judgment that leads to an action (merge, reject, or feedback) — deliberately defined so the 'entity' may be an agent and so review is distinguished from casual reading.
Shift Left
Moving deterministic verification (formatting, type errors, dependency policy, schema checks) as early as possible — into the IDE, pre-commit, or pre-push — so that only genuinely judgment-requiring concerns reach review.
Context Engineering (as applied to review) Context Engineering
Deliberately making a codebase 'agent-ready' by turning it into a repository of operational knowledge, architectural intent, and historical decisions (AGENTS.md, knowledge graphs, indexed incident/postmortem history) — not an optional add-on but infrastructure every other review pattern depends on.
Adversarial LLM Review
Structural independence between the agent that writes code and the agent that reviews it (different model, different session with no shared context, or structured adversarial prompting) — modeled explicitly on GANs (generator/discriminator) and Zero Trust (explicit verification, least privilege).
Multi-Agent Reviews
Decomposing review into specialist agents (security, performance, compliance, standards, correctness, operational readiness) coordinated by a triage agent (selects which specialists run) and a coordinator agent (deduplicates and synthesizes findings into one structured assessment) — an ensemble-learning analogy applied to review.

Propositions

explicitManual, human-only code review cannot scale with AI-generated code volume, but the solution is neither simply adding more human reviewers nor removing humans entirely — it is redistributing review into a layered, continuous, risk-tiered system (deterministic checks, author/review-agent loops, adversarial and multi-agent AI review, selective human judgment).conf: high
extendsAgentic Code Review · p12 — Both describe the human moving 'up a level' from reading every diff to owning accountability/judgment; this paper operationalizes that shift into nine named, composable patterns and three real deployments.
explicitcorroboratedCode review's value substantially exceeds defect-finding: citing a Microsoft study and two further empirical papers, most review activity addresses maintainability, style, alternate-solution discovery, and knowledge sharing — 'evolvability' risk — rather than catching functional bugs, which means any redesign of review must preserve these non-defect functions, not just correctness checking.conf: high
supports ←Bugs in Large Language Models Generated Code: An Empirical Study · p8 — If a meaningful share of LLM bugs require semantic judgment rather than pattern-matching against known human mutant operators, this reinforces the claim that review's value (and now testing's value) lies beyond simple defect-pattern matching — in catching maintainability, intent-fit, and non-obvious category errors.
explicit'No human in the loop' should be a deliberate, risk-and-evidence-justified configuration choice, never a default, and every agentic review decision must produce a durable, inspectable record (what was evaluated, under which policy, with what confidence, by whom) to preserve organizational accountability.conf: high
extendsAgentic Code Review · p13 — Directly answers the 'borrowed confidence' risk of a closed self-reviewing loop sharing blind spots: this paper's requirement for durable, inspectable audit records plus risk-tiering is the proposed structural remedy.
explicitAI code generation, left to its defaults, produces large synthesized change sets that overwhelm human and AI reviewers alike, so keeping changes small becomes more important under AI, not less — because large diffs are harder for AI reasoning to evaluate too, not only for human attention.conf: high
supportsGitHub Brings Stacked Pull Requests to Public Preview · p2 — Both identify shrinking change-set scope (stacked/incremental PRs) as the lever that reduces the review bottleneck, rather than simply demanding reviewers process more per unit time.
explicitdevelopedReviewing code with the same model/session/assumptions as the agent that wrote it reproduces the author's own blind spots; adversarial review requires structural independence (a different model, a context-isolated session, or structured adversarial prompting) modeled on GAN generator/discriminator dynamics and Zero Trust's explicit-verification and least-privilege principles.conf: high
supportsAgentic Code Review · p10 — Corroborates the recommendation to run heterogeneous/independent reviewers: the cited experiment found minimal overlap between four review tools' findings, which is the concrete evidence this paper's Adversarial LLM Review pattern argues for by analogy.
extends ←How I write software with LLMs · p5 — Where the other document frames adversarial, structurally-independent review theoretically (via the GAN/Zero-Trust analogy), this article supplies a concrete, named production workflow (architect/developer/1-3 reviewers, escalation on disagreement) that instantiates the same principle.
explicitAgents lack the tacit institutional memory (hallway conversations, accumulated intuition) that lets human reviewers catch things like an undocumented client dependency; organizations that skip deliberate Context Engineering find their agentic-review failures trace to missing organizational memory, not insufficient model intelligence.conf: high
supportsGit real: AI agents aren’t just for solo developers anymore · p3 — Both treat remembering organizational decisions across people and sessions as a team-scale requirement that a personal-scale harness or review setup doesn't need, reinforcing Context Engineering as infrastructure, not configuration.
explicitdevelopedSpec-Driven Development relocates the primary review gate from implementation to the plan: once intent is reviewed and hardened up front, code review's job shifts from trying to catch what went wrong to verifying that a trusted, already-approved plan was faithfully executed — because by the time code exists, the most consequential (and hardest to reverse) decisions have already been made.conf: high
extends ←10 Things Developers Want from their Agentic IDEs in 2025 · p7 — This piece names developers' desire for spec-driven artifacts as durable, session-surviving contracts; the other extraction specifies the mechanism — review relocates from implementation to the plan once the spec is hardened.
implicitAcross three case studies (Block, Cloudflare, DryRun Security) operating under very different constraints (internal AI-first scale-up, CI-native infrastructure vendor, AI-native security startup), each organization converged independently on the same layered-escalation shape — shift-left, small increments, continuous author/reviewer agent loop, then adversarial or multi-agent review — because the same underlying constraint (code generation is cheap, organizational attention is not) forces the same architecture regardless of starting point.conf: medium
explicitcorroboratedAs agentic systems absorb mechanical code generation and review, professional role boundaries blur (developer/tester, and now architect/developer/product manager), and the decisive remaining human skill shifts toward writing precise specifications, designing risk-tiered verification policy, and exercising judgment on ambiguity and strategic fit — not toward reviewing more code faster.conf: high
supports ←Bugs in Large Language Models Generated Code: An Empirical Study · p10 — The taxonomy's split between prompt-caused categories (Misinterpretation, Missing Corner Case — failures of ambiguous intent) and generatively-caused categories (Hallucinated Object, Wrong Attribute) gives empirical grounding to the claim that the decisive human skill is writing precise specifications and judging ambiguity, not mechanically re-checking code.

Arguments

Human-only, single-gate code review cannot be preserved as AI accelerates code volume; it must be redistributed into a layered, continuous, risk-tiered verification system. strength: high

Explicit premises:

Senior engineers already received dozens of merge-review requests per day before full agentic adoption.
AI further multiplies code volume and generation rate.
Human code review persists today because of three interlocking forces — trust, compliance, and accountability — not because it is uniquely effective at finding defects.

Implicit premises:

Whatever replaces single-gate human review must preserve review's non-defect functions (knowledge sharing, risk triage, maintainability signal), not just correctness checking.
Neither extreme — status quo manual review, nor fully autonomous review with no human — satisfies trust, compliance, and accountability simultaneously.

Inferences:

The resulting system (deterministic shift-left + continuous author/reviewer loop + adversarial/multi-agent AI review + human escalation for ambiguity and accountability) is close to a forced architecture given these constraints, not a stylistic preference.
Organizations that try to resist the change (pure manual review) or go fully autonomous (no durable accountability trail) are both predicted to fail — illustrated by Block's initial failed homegrown AI review and by every case study's retention of a human escalation path.

Independent, structurally adversarial review is necessary wherever an author agent and a reviewer agent might otherwise share assumptions, because shared-context review cannot catch shared blind spots. strength: medium