Jagged Capability (On Rails / Off Rails)

The 'jagged frontier' (coined in Dell'Acqua, Mollick et al.'s 2023 HBS/BCG study; later popularised by Karpathy) — LLM competence as domain-dependent and extreme — brilliant, PhD-level performance in verifiable, RL-optimized domains ("on rails") coexisting with basic failures elsewhere ("off rails") — a variance far more extreme than in humans.

Also known as: jaggedness, on rails / off rails, jagged frontier

Part of: LLM Limitations

How authors use it

Oldest first (extraction date when no publication date is known).

Documents (5)

• The Problem Is Prompt Debt Drew Breunig · 2026-07-30 source ↗ Reader 🖥️
  Because natural language is an imprecise specification language for a probabilistic model, hand-tuning prompts to define durable system behavior accumulates a distinct 'prompt debt' — slowing iteration, becoming illegible to teams, and locking the application to one model's idiosyncrasies — which can only be paid down by specifying behavior with measurable evals rather than prose and letting automated search (e.g. DSPy, GEPA) generate and optimize the prompt itself.
• Co-Existence and the End of Co-Intelligence Ethan Mollick · 2026-06-04 source ↗ Reader 🖥️
  The tight, human-centered, back-and-forth 'co-intelligence' mode of working with AI that Mollick described in 2023 is giving way to a 'co-existence' era of greater AI autonomy — sometimes better than you, sometimes not — in which the real skill is a continuously renegotiated judgment about when to lean on AI, when to refuse it, and when to hand over control entirely, extending even to AI now acting as one's reader, critic, and gatekeeper.
• LLMs Can't Jump Tom Zahavy, Google DeepMind · 2026-05-23 source ↗ Reader 🖥️
  Modern LLMs have mastered Induction (statistical pattern-matching) and are rapidly mastering Deduction (formal proof), but are structurally incapable of Abduction — the embodied-simulation-grounded leap from scarce sense experience to novel explanatory axioms that produced discoveries like General Relativity — so genuine AI scientific invention requires physically-consistent, interactive, multimodal world models rather than more language-model scale.
• Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI No Priors: AI, Machine Learning, Tech, & Startups · 2026-03-20 Reader 🖥️
  The jump in code-agent capability since December 2024 has made human throughput — not model capability — the binding constraint, and the central challenge is now removing yourself from the loop so autonomous agents can maximize token throughput on your behalf.
• Co-Intelligence, by Ethan Mollick: a Summary Jérémie Rostan · 2024-11-18 source ↗ Reader 🖥️
  Large language models are best understood not as traditional software but as an unpredictable, human-like 'alien intelligence' with jagged, unevenly distributed capabilities — which means working with them well requires deliberate, continuously renegotiated human practices (experimentation, oversight, personification, assuming today's AI is the worst you'll ever use) across creative work, the workplace, and education, in service of augmenting rather than replacing human intelligence, judgment, and meaning.

Propositions touching this concept (2)

By publication month, oldest first (extraction month when no publication date is known).

2026-06

• explicitdevelopedAI remains far from perfect and its competence is unevenly distributed (the jagged frontier), leaving substantial room for humans not just to use AI but to thrive by working alongside it.
  Co-Existence and the End of Co-Intelligence · p4

2026-07

• implicitcontestedThe article assumes that lessons from coding-agent practitioners — a narrow, code-generation-specific population at the 'jagged frontier' — transfer cleanly to specifying AI application behavior in general, even though coding and general application behavior may differ in how measurable or evaluable they are.
  The Problem Is Prompt Debt · p14

Open gaps (11)

• incompleteAuto research is demonstrated only on hyperparameter optimization — a narrow, well-defined search with clear metrics. No evidence or discussion of how it would work on research requiring genuine novelty, taste, or paradigm shifts.
  Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI — read next: Literature on AI-driven scientific discovery beyond optimization: can agents formulate novel hypotheses, not just test existing ones? See work on AI scientists (e.g., Sakana AI's 'The AI Scientist').
• uninformedThe Jevons paradox argument for software engineering jobs is applied without discussing the significant disanalogies: ATMs automated a narrow task within banking, while AI agents potentially automate the entirety of software production. Demand elasticity analysis is hand-waved.
  Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI — read next: Economic analysis of AI labor displacement that distinguishes between task-level automation (Jevons applies) and occupation-level automation (potentially different dynamics). See Acemoglu vs Brynjolfsson debate.
• incompleteSecurity and trust implications of the 'apps dissolve into APIs' vision are acknowledged but not seriously explored. Dobby discovered unprotected Sonos APIs — this is a security vulnerability presented as a feature.
  Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI — read next: Research on agent-to-API security models, OAuth for agents, and the attack surface of agent-mediated home automation.
• illogicalKarpathy simultaneously claims (1) everything is a skill issue and current tools are sufficient, and (2) models are jagged, burst at the seams, and going too far ahead makes them net negative. These are in tension — if the tools truly suffice, jaggedness should be manageable.
  Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI — read next: Framework for distinguishing 'skill issues' from genuine capability boundaries in AI agent systems.
• incompleteThe paper's positive proposal -- that action-controllable, physically-consistent multimodal world models will supply the sensory grounding needed for abduction -- is asserted but not operationalized. No benchmark, no success criterion, and critically, no account of how a 'counterfactual physical simulation' would itself get translated into a formal axiom: translation-from-simulation-to-axiom is exactly the bottleneck the paper identifies as the Jump, and it recurs unaddressed inside the proposed solution rather than being solved by it.
  LLMs Can't Jump — read next: Genie / Genie 2 / Genie 3 technical reports and any successor work on translating interactive world-model states into formal symbolic theories; LeCun's JEPA and world-model architecture papers for how (or whether) symbol grounding is actually operationalized there.
• incompleteThe entire argument rests on one historical case (Einstein/General Relativity), reconstructed via a single secondary source (Norton, 2020). No comparison is made to other major discoveries made under data scarcity -- Darwin's natural selection, Wegener's continental drift, Mendel's genetics -- to test whether 'manipulative abduction via embodied simulation' is the general mechanism of discovery-under-scarcity or an idiosyncrasy of how Einstein specifically thought (he is quoted saying words play no role in his mechanism of thought -- a documented personal trait, not obviously evidence about scientific discovery in general).
  LLMs Can't Jump — read next: Case studies of scientific discovery in non-physics domains (biology, geology) under comparable data scarcity, to test whether the embodied-simulation account generalizes or is Einstein-specific.
• illogicalThe paper concedes that an AI could plausibly identify the conceptual inconsistency between Newtonian absolute time and Maxwell's constant speed of light that motivated Einstein's search in the first place -- an act of noticing which surprising Result is worth explaining. Yet the paper classifies this act as merely deductive ('much like a system identifying buggy code'), while its own Peircean framework defines exactly this kind of selection (choosing which Case merits explanation) as characteristically abductive. This blurs the sharp induction/deduction/abduction boundary the paper otherwise insists on, without acknowledging the tension.
  LLMs Can't Jump — read next: Peirce's own writing on abduction as hypothesis-selection (not just hypothesis-generation) would clarify whether 'noticing an inconsistency worth resolving' counts as abduction under his original scheme.
• incompleteThe claim that 'what is happening in coding is going to be happening in many fields' is asserted by analogy immediately after the coding statistics, with no argument for why the pattern should generalize. Coding benefits from dense training data and cheap, automatic ground-truth verification (compilers, tests) that many other knowledge-work domains lack.
  Co-Existence and the End of Co-Intelligence — read next: The corpus already contains a direct counter-argument on this exact point (extract-breunig-the-problem-is-prompt-debt#p14: lessons from coding-agent practitioners at the 'jagged frontier' don't transfer cleanly to less verifiable domains) and evidence that coding-agent gains themselves attenuate through the delivery pipeline (extract-five-studies-ai-software-engineering#p4).
• incompleteThe book (per this summary) does not substantively address AI's environmental cost — resource extraction, energy consumption, CO2 emissions — despite being otherwise comprehensive across societal, educational and workplace impacts. The summary names this explicitly as a blind spot rather than the extraction inferring it.
  Co-Intelligence, by Ethan Mollick: a Summary — read next: Material on AI's compute/energy footprint and its organizational and policy implications.
• incompleteThis corpus's own concept registry currently attributes the 'Jagged Frontier' term to Karpathy, but this summary's source material grounds it explicitly in Mollick's own (co-authored) HBS research, which the book popularizes — the registry description may need correcting or broadening to credit Mollick/Dell'Acqua et al. as the term's origin, distinct from whatever later use Karpathy made of it.
  Co-Intelligence, by Ethan Mollick: a Summary — read next: Check the original HBS 'jagged frontier' paper (Dell'Acqua et al., cited directly in this summary) against whatever Karpathy source the registry entry currently draws on.
• incompleteMollick's 'Centaur' (a strategy for dividing labor with AI, paired with 'Cyborg') and Cory Doctorow's 'centaur' (a benefit-direction diagnostic, paired with 'reverse centaur') are two unrelated concepts that happen to share a word — a textbook term-translation hazard for syntopical reading across this corpus, since a naive keyword match would wrongly treat all 'centaur' references as the same idea.
  Co-Intelligence, by Ethan Mollick: a Summary — read next: extract-everyone-i-recommend-on-ai-is-optimistic-except-cory-doctorow (Doctorow's centaur/reverse-centaur) — read side by side with this document's Centaur/Cyborg to keep the two senses distinct in the registry and in any term-translation table.