GitHub Copilot CLI combines model families for a second opinion
Nick McKenna · article · 2026-02 source ↗ Reader 🖥️
A single coding agent reviewing its own work is limited by its training biases; pairing it with a second model from a different AI family at key checkpoints closes most of the quality gap to a stronger model, catching architectural errors, subtle bugs, and cross-file conflicts that self-reflection alone misses.
practical · product announcement / technical blog post — Announces and explains Rubber Duck, a cross-model review feature in GitHub Copilot CLI, combining product marketing with a technical rationale grounded in evaluation data.
Structure
| Part | Role |
|---|---|
| The problem: Confident mistakes can compound | Establishes the failure mode — early planning errors cascade, and self-review is bounded by shared training biases. |
| Rubber Duck adds a second perspective | Introduces the solution — a cross-family review agent — and defines what it does. |
| When does the cross-family review help? (SWE-Bench Pro evaluation) | Provides quantitative evidence: Sonnet + Rubber Duck closes 74.7% of the gap to Opus. |
| Concrete examples (OpenLibrary, NodeBB) | Illustrative narratives showing the kinds of errors caught — architectural, one-liner, cross-file. |
| When does Rubber Duck activate? | Describes the invocation design: after plan, after implementation, after tests, reactively when stuck, and on-demand. |
| Getting started / CTA | Product marketing — how to enable and use the feature. |
Key terms
- Rubber Duck Multi-Agent Orchestration
A focused review sub-agent powered by a model from a different AI family than the primary orchestrator, invoked at key checkpoints to critique the agent's plan and work. - cross-family review Multi-Agent Orchestration
Having a model from a different training lineage (e.g., GPT-5.4 reviewing Claude's work) to surface blind spots that same-family self-reflection cannot. - training biases LLM Reliability Failures
Systematic blind spots arising from a model's training data and techniques that persist even during self-reflection. - orchestrator Multi-Agent Orchestration
The primary model driving the coding agent session, selected by the user from the model picker. - checkpoint
A moment in the agent loop (after planning, after implementation, after test-writing) where a review has the highest return on investment.
Propositions
- explicitcorroboratedA model reviewing its own work is bounded by its own training biases — same data, same techniques, same blind spots.conf: highsupports ←How Coders Can Survive—and Thrive—in a ChatGPT World · p9 — Vaithilingam's warning that LLMs hallucinate and produce subtly wrong code that is hard to spot anticipates the later, more precise diagnosis that a model (or its own self-review) is bounded by the same training biases and blind spots that produced the error in the first place.
- explicitClaude Sonnet 4.6 + Rubber Duck (GPT-5.4) closes 74.7% of the performance gap between Sonnet and Opus on SWE-Bench Pro.conf: high
- explicitRubber Duck helps more on difficult problems — 3+ files, 70+ steps — with a 3.8% improvement over baseline and 4.8% on the hardest problems.conf: high
- explicitcorroboratedEarly planning errors compound downstream; catching them at the plan stage yields the biggest wins.conf: highrefines ←My AI Kept Pushing Me to Ship, So I Asked It Why · p6 — Refines the cross-family-diversity account of a reliable AI 'second opinion': a fresh instance of the *same* model family, with no conversational momentum or stake in the prior framing, also provides a meaningfully more objective read — suggesting freshness/statelessness, not just architectural diversity, is doing real work.supports ←The Advisor Pattern: Combining Fast Models with Smart Guidance for Scalable AI Systems · p7 — Rubber Duck's SWE-Bench Pro data (biggest gains from catching errors at the planning stage) is empirical evidence for the same 'call the advisor early, before firm decisions' claim this article only asserts without data.
- implicitcorroboratedCross-family model diversity is a more effective form of 'second opinion' than same-model self-reflection.conf: highrefinesAgents don't know what good looks like. And that's exactly the problem. · p2 — Cross-family review partially mitigates a single model's inability to recognize its own invalid solutions, qualifying the design-constraint claim.supports ←Have your agent record video demos of its work with shot-scraper video · p9 — Willison's habitual Anthropic-reviews-OpenAI-and-vice-versa practice is direct practitioner corroboration that cross-family diversity beats same-model self-reflection.extends ←GitHub Copilot's Project HydraFusion Promises Frontier Level Performance Through Multi-Model Routing · p3 — HydraFusion's Critique execution pattern productizes cross-family review as one of three formal runtime modes in a shipping feature, building on the earlier-reported Rubber Duck finding that cross-family model diversity beats same-model self-reflection.
- implicitThe quality improvement from cross-family review is cost-effective because Rubber Duck is invoked sparingly at high-signal moments rather than continuously.conf: medium
- implicitModel families trained by different organizations have sufficiently different blind spots that one can reliably catch errors the other misses.conf: medium
Arguments
Cross-family model review (Rubber Duck) meaningfully improves coding agent quality. strength: medium
Explicit premises:
- Self-reflection is bounded by same-family training biases.
- Sonnet + Rubber Duck closes 74.7% of the Sonnet-to-Opus gap on SWE-Bench Pro.
- The improvement is larger on harder, multi-file problems.
Implicit premises:
- Models from different families have complementary (non-overlapping) blind spots.
- SWE-Bench Pro is a representative proxy for real-world coding quality.
- Opus alone is the appropriate quality ceiling for comparison.
Inferences:
- Therefore a cheaper model + cross-family review approaches the quality of a more expensive model alone.
Review should happen at specific checkpoints rather than continuously. strength: medium
Explicit premises:
- Feedback has highest return after planning, after complex implementation, and after writing tests.
- Invoking Rubber Duck sparingly avoids getting in the way.
Implicit premises:
- Continuous review would be too slow or too expensive.
- The marginal value of review diminishes rapidly outside these checkpoints.
Inferences:
- Therefore a sparse, targeted invocation strategy is optimal.
Narratives
- Rubber Duck caught an async scheduler in OpenLibrary that would start and immediately exit, running zero jobs — and that even if fixed, one of the tasks was itself an infinite loop.
→ Illustrates the 'architectural catch' category — errors in design that cascade if undetected. - Rubber Duck caught a loop silently overwriting the same dict key on every iteration, dropping 3 of 4 Solr facet categories from search queries with no error thrown.
→ Illustrates the 'silent bug' category — small code errors with outsized impact and no visible failure signal. - Rubber Duck caught three files reading from a Redis key that the new code stopped writing, which would have silently broken the confirmation UI and cleanup paths on deploy.
→ Illustrates the 'cross-file conflict' category — changes whose consequences span multiple files and are invisible at the local level.
Questions addressed
- ✅ Why does a coding agent sometimes produce confidently wrong plans?
Because early planning errors compound, and self-reflection cannot escape the model's own training biases. - ✅ Can a second model from a different family materially improve agent quality?
Yes — Sonnet + Rubber Duck closes 74.7% of the gap to Opus on SWE-Bench Pro. - ✅ When should the second opinion be invoked?
After planning, after complex implementation, after writing tests, reactively when stuck, and on user demand. - ⭕ What is the cost/latency overhead of cross-family review?
Not addressed. The article says Rubber Duck is invoked 'sparingly' but provides no data on added latency, token cost, or wall-clock time.
Gaps
- missing evidenceThe claim that models from different families have complementary blind spots is the core theoretical justification, but no analysis of what specific biases differ between Claude and GPT families is provided.
Read next: Look for research on training-data or RLHF-induced biases across model families; compare failure modes on shared benchmarks. - missing dataNo cost, latency, or token-usage data is provided for Rubber Duck invocations. The economic argument (cheaper model + review ≈ expensive model alone) is implied but unquantified.
Read next: Benchmark Sonnet + Rubber Duck against Opus on wall-clock time and total token cost per task. - selection biasAll three illustrative narratives are success stories. No examples of Rubber Duck giving unhelpful, misleading, or contradictory feedback are shown.
Read next: Look for cases where Rubber Duck's feedback degraded the primary agent's output or caused unnecessary rework. - scope limitationSWE-Bench Pro is the only benchmark cited. Generalization to other coding tasks (greenfield development, non-Python codebases, design-heavy work) is unstated.
Read next: Test Rubber Duck on diverse task types beyond bug-fix benchmarks. - unexplored alternativeThe article does not compare cross-family review against other quality-improvement techniques: multiple same-family samples (majority voting), structured self-critique prompts, or human review at the same checkpoints.
Read next: Compare Rubber Duck to best-of-N sampling with the same model or to chain-of-verification with explicit critique prompts.
Related documents
- Agents don't know what good looks like. And that's exactly the problem.
- Have your agent record video demos of its work with shot-scraper video
- Agentic AI and Code Reviews
- Accelerating Delivery: How AI Agents Can Own the Initial Code Draft
- [FULL WORKSHOP] AI Coding For Real Engineers - Matt Pocock, AI Hero
- Android Studio Quail 2 Redesigns Agent Mode, Streamlines AI-Assisted Coding
- Bun Rewrites 535K Lines of Zig into Rust in Four Months, Eliminates Numerous Memory Leaks
- Jacob Langvad Nilsson's Post
- How Datadog Used Claude and Cursor for Test-Driven Production Migration
- Why AI Coding Agents Still Need Clear Specs