LLMs Can't Jump
Tom Zahavy, Google DeepMind · pdf · 2026-05-23 source ↗ Reader 🖥️
Abductive Reasoning & the Limits of AI DiscoveryPhysically-Grounded / Embodied World ModelsJagged Capability (On Rails / Off Rails)LLM LimitationsPattern-Completion Generation (Not Reasoning)
Modern LLMs have mastered Induction (statistical pattern-matching) and are rapidly mastering Deduction (formal proof), but are structurally incapable of Abduction — the embodied-simulation-grounded leap from scarce sense experience to novel explanatory axioms that produced discoveries like General Relativity — so genuine AI scientific invention requires physically-consistent, interactive, multimodal world models rather than more language-model scale.
theoretical · argumentative — A position paper structured as sustained argument: it adopts Peirce's induction/deduction/abduction trichotomy and Einstein's E-J-A model as a framework, then marshals a detailed historical case study (Einstein's path to General Relativity) as evidence for a claim about what LLMs structurally cannot do. It is theoretical throughout — arguing 'X is true' (LLMs lack abduction) rather than 'you should do Y' — though it closes with a practical-sounding research proposal (build interactive world models) that is itself justified theoretically, not argued as an action imperative in its own right.
Structure
| Part | Role |
|---|---|
| Introduction: Einstein's letter to Solovine (Sense Experience -> Jump -> Axioms -> Deduction -> Verification) and Peirce's Rule/Case/Result trichotomy (Deduction, Induction, Abduction). | Framework-setting; states the thesis and the analytic tool used throughout. |
| Background (Section 2): the crisis in 19th-century mechanics, Special Relativity (1905), and the three-phase reconstruction of General Relativity's discovery (Ideation 1907-12, Consolidation 1912-13 incl. the 'Fatal Error', Mathematical validation 1913-15) via Norton (2020). | Evidence / case study — establishes the historical facts the rest of the paper argues from. |
| Section 3, The Limits of Inductive Inference: refutes Schmidhuber's 'creativity as compression' as an account of Einstein's discovery, on the grounds that Newtonian gravity faced no empirical crisis and confirming data arrived only after the theory was formulated. | Refutation of rival theory 1 (induction accounts for discovery). |
| Section 4, The Limits of Deduction: concedes a modern LLM/proof system (AlphaProof-style) could plausibly re-derive General Relativity's field equations given Einstein's axioms, but argues deduction is strictly downstream verification and cannot generate the axioms themselves. | Refutation of rival theory 2 (deduction accounts for discovery), while conceding LLM competence at deduction specifically. |
| Section 5, Abduction: The Missing Jump: positive account via Magnani's 'manipulative abduction' and Einstein's elevator/falling-observer thought experiment; frames LLMs as 'Chinese Rooms' lacking sensory grounding; distinguishes passive world models (Veo) from action-controllable ones (Genie); proposes physically-consistent, multimodal, interactive world models as the path to closing the gap. | Main positive thesis and forward-looking proposal. |
| Conclusion: restates the negative verdict for current LLMs, situates AlphaEvolve/AI Scientist as still lacking the abductive gap-closer, and scopes the claim to the physical sciences (vs. mathematics/CS, gestured at but not developed). | Synthesis and scope-limitation. |
Key terms
- The Jump (J) Abductive Reasoning & the Limits of AI Discovery
The intuitive, non-logical cognitive leap Einstein described in his letter to Solovine connecting Sense Experience (E) to a System of Axioms (A) — the specific act the paper argues LLMs cannot perform. - Abduction Abductive Reasoning & the Limits of AI Discovery
Following Peirce: inference of a Case (or new Rule) from a Rule and a surprising Result (Rule+Result -> Case) — a creative leap that invents a cause for a singular phenomenon, distinct from both deduction (guarantees truth) and induction (finds statistical patterns). - Manipulative Abduction Abductive Reasoning & the Limits of AI Discovery
Magnani's (2009) concept: abduction driven by active, embodied interaction with mental models ('thinking by doing') rather than passive symbolic search — e.g., Einstein simulating the physical sensation of falling. - Creativity as Compression Abductive Reasoning & the Limits of AI Discovery
Schmidhuber's (2008) theory that scientific discovery is driven by the inductive desire to compress data — the 'joy' of discovery being the rate at which complex observations become subjectively simpler. Used in this paper as the rival theory being refuted. - Chinese Room Physically-Grounded / Embodied World Models
Applied via Harnad's symbol-grounding framing: LLMs manipulate the language of physics fluently without access to the physical referents that give that language meaning — high-dimensional but ungrounded. - Physically-Grounded World Model Physically-Grounded / Embodied World Models
Physically-consistent, multimodal, action-controllable generative environments (e.g., Genie) that allow counterfactual intervention ('take control of the simulation') — explicitly distinguished from passive video-prediction models (e.g., Veo) that merely reproduce statistically dominant visual continuations.
Propositions
- explicitLLMs have mastered Induction (statistical pattern matching from data) and are rapidly mastering Deduction (formal derivation/proof, e.g. AlphaProof, Lean-based theorem proving), but are structurally incapable of Abduction — the generation of novel explanatory hypotheses or axioms.conf: highextendsPhilosophy Eats AI: What Leaders Should Know · p3 — Zahavy's Peircean framing gives 'philosophy eats AI's' claim that GenAI 'runs on pattern similarity and relevance, not logic or reason' a precise mechanism (Induction) and sharpens it further: deduction is also within reach, but abduction specifically is not.
- explicitThe 'creativity as data compression' theory of discovery (Schmidhuber) fails to explain the invention of General Relativity because there was no statistically significant, error-rich dataset to compress: Newtonian gravity was verified to 10^-9 precision and its one anomaly (Mercury's perihelion) was patched with the ad hoc 'Vulcan' hypothesis rather than treated as a falsification signal.conf: high
- explicitDeduction is a strictly downstream/verification process: it can derive consequences from given axioms but cannot generate the axioms themselves, so a modern LLM initialized with Einstein's 1915 assumptions could plausibly re-derive the General Relativity field equations, but this says nothing about whether it could have invented them.conf: high
- explicitThe critical bottleneck is translating embodied, simulated physical experience into formal axioms, exemplified by Einstein's 'happiest thought' (the falling-observer/elevator thought experiment), which relied on manipulative abduction — grounding abstract symbols in simulated sensory experience rather than manipulating pre-existing symbols.conf: high
- explicitLLMs operate as high-dimensional 'Chinese Rooms', manipulating the language of physics fluently without access to the physical referents that give that language meaning, so they lack the sensory agency required for the abductive Jump.conf: high
- explicitPassive video-generation world models (e.g. Veo) exhibit 'intuitive physics' only as a statistical byproduct of the training distribution, not because they causally model gravity; genuine grounding requires action-controllable, counterfactually-interactive world models (e.g. Genie-style) that let an agent 'take control of the simulation'.conf: high
- implicitVerifiable, RL-optimizable domains (formal proof, code) are where LLM capability gains concentrate, while abduction-requiring domains structurally lack a clean error/reward signal — meaning current RL-based scaling recipes cannot, by their own logic, be expected to close the abductive gap.conf: mediumsupportsSkill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI · p8 — Zahavy's account of deduction (verifiable domains, e.g. AlphaProof) versus abduction (no error signal available in Einstein's case) gives a structural/epistemological explanation for Karpathy's empirical observation that RL-driven LLM gains are limited to verifiable domains and don't generalize to non-verifiable ones.
- implicitThe author treats a single historical case (Einstein's personal cognitive style, including his own remark that 'the words... do not seem to play any role in my mechanism of thought') as sufficient grounds for a general claim about the cognitive architecture required for scientific discovery under data scarcity as such.conf: medium
Arguments
Modern LLMs are currently structurally incapable of the abductive Jump required for genuine scientific invention (as distinct from formal derivation from given premises). strength: medium
Explicit premises:
- Peirce's trichotomy (deduction, induction, abduction) structurally exhausts the modes of inference relevant to hypothesis generation.
- LLMs demonstrably excel at induction (statistical pattern matching) and are rapidly improving at deduction (AlphaProof, Lean-based theorem proving).
- Abduction, per Magnani, requires 'manipulative' embodied simulation -- active sensorimotor interaction with mental models -- not passive symbol manipulation.
- LLMs lack sensory agency and embodied interaction with physical reality; they operate as 'Chinese Rooms' over text.
Implicit premises:
- Embodied/sensory grounding is necessary (not merely one sufficient route among several) for generating genuinely novel axioms under data scarcity.
- Text-based training data cannot substitute for embodied simulation, because the relevant meaning (e.g. the felt equivalence of acceleration and gravity) did not exist in symbolic form prior to the discovery it enabled.
Inferences:
- If future interactive, physically-consistent, multimodal world models (successors to Genie) achieve sufficient fidelity and counterfactual controllability, the bottleneck would be an engineering/architecture problem rather than a permanent metaphysical one -- implying the paper's own thesis is falsifiable by future systems, though no criterion for when it would count as falsified is given.
The prevailing 'creativity as compression' account of discovery cannot explain the invention of General Relativity. strength: high
Explicit premises:
- Compression-based induction requires a supervised error signal, i.e. data whose compression is being sought.
- At the time of Einstein's invention, Newtonian gravity faced no empirical crisis -- verified to 10^-9 precision, with its one anomaly (Mercury's perihelion) explained away via the ad hoc 'Vulcan' hypothesis rather than treated as falsification.
- The empirical evidence that did confirm General Relativity (the Eddington eclipse experiment, relativistic GPS corrections) arrived only after the theory was formulated.
Implicit premises:
- An inductive/compression-driven system, absent a strong error signal, has no gradient that would drive it toward a more complex, costlier hypothesis (non-Euclidean geometry) over a cheaper local patch (Vulcan).
- The retrospective elegance/simplicity of a finished theory is not evidence that the discovery process which produced it was itself compression-driven.
Inferences:
- The same logic extends the critique beyond LLMs specifically to any RL-based 'AI Scientist' system that depends on a reward/error signal -- which the paper makes partially explicit regarding Sakana's AI Scientist and Google DeepMind's AlphaEvolve in the Conclusion.
Narratives
- Einstein's roughly seven-year path to General Relativity: Ideation (1907-1912, the 'happiest thought'), Consolidation (1912-1913, the Zurich Notebook with Grossmann and the 'Fatal Error' of discarding the correct Riemann-tensor-derived equations), and Mathematical validation (1913-1915, culminating in the four-week sprint against Hilbert in November 1915).
→ Shows, rather than abstractly argues, that discovery was effortful, error-prone, non-monotonic search under genuine uncertainty -- grounding the Induction/Deduction/Abduction taxonomy in a textured historical process instead of an idealized reconstruction, and supplying the concrete evidence for p2 and p3. - The 'happiest thought' -- Einstein imagining a physicist inside an accelerating elevator/free-falling from a roof, where released objects appear to fall together regardless of composition, leading him to abduct that acceleration and gravity are the same phenomenon.
→ The paper's central illustration of manipulative abduction: an axiom (the Equivalence Principle) generated from simulated bodily/sensory experience rather than from data or logic. Directly supports p4. - The 'Vulcan' hypothesis: rather than doubting Newtonian gravity when Mercury's perihelion anomaly was observed, scientists posited an undiscovered planet near the Sun to explain it away.
→ Illustrates how an inductive/compression-minded system would behave under a weak anomaly signal -- patching locally rather than restructuring globally -- used as the paper's concrete counter-example to what a compression-driven AI would likely do when faced with the same data. Supports p2 and the implicit premise in Argument 2. - Figure 1: a 'generative AI reconstruction' of Einstein's own E-J-A diagram from the Solovine letter, in which the AI hallucinates/garbles the axiomatic symbols.
→ A self-demonstrating irony the author calls out explicitly: the AI's own failure to faithfully reproduce a diagram of the very mechanism (the Jump) it lacks is offered as a small performative illustration of the paper's thesis.
Questions addressed
- ✅ Could a modern AI, given the knowledge available to Einstein, invent General Relativity?
No, for the abductive/invention phase specifically -- though yes, plausibly, for the deductive derivation phase if the axioms are supplied as inputs. - ✅ What cognitive mechanism allowed Einstein to generate new axioms (e.g. the Equivalence Principle) in the absence of sufficient symbolic/textual data?
Manipulative abduction via embodied simulation -- grounding abstract symbols in the simulated physical sensation of a thought experiment. - ⭕ How does the paper's embodied-simulation framework generalize beyond physics, to domains like mathematics or computer science where 'Sense Experience' is not physical?
Gestured at only in the closing paragraph ('for mathematics, it is the abstract landscape of formal systems') without working out what embodied/manipulative abduction would even mean absent a body to simulate with. - ⭕ What would count as evidence that a future interactive world model has actually closed the abductive gap, as opposed to merely producing more convincing simulacra of discovery?
Not addressed -- the paper proposes world models as the path forward but, unlike its treatment of the deductive/logical leap (for which it cites ARC-AGI as an existing benchmark), specifies no benchmark or success criterion for abduction.
Gaps
- incompleteThe paper's positive proposal -- that action-controllable, physically-consistent multimodal world models will supply the sensory grounding needed for abduction -- is asserted but not operationalized. No benchmark, no success criterion, and critically, no account of how a 'counterfactual physical simulation' would itself get translated into a formal axiom: translation-from-simulation-to-axiom is exactly the bottleneck the paper identifies as the Jump, and it recurs unaddressed inside the proposed solution rather than being solved by it.
Read next: Genie / Genie 2 / Genie 3 technical reports and any successor work on translating interactive world-model states into formal symbolic theories; LeCun's JEPA and world-model architecture papers for how (or whether) symbol grounding is actually operationalized there. - incompleteThe entire argument rests on one historical case (Einstein/General Relativity), reconstructed via a single secondary source (Norton, 2020). No comparison is made to other major discoveries made under data scarcity -- Darwin's natural selection, Wegener's continental drift, Mendel's genetics -- to test whether 'manipulative abduction via embodied simulation' is the general mechanism of discovery-under-scarcity or an idiosyncrasy of how Einstein specifically thought (he is quoted saying words play no role in his mechanism of thought -- a documented personal trait, not obviously evidence about scientific discovery in general).
Read next: Case studies of scientific discovery in non-physics domains (biology, geology) under comparable data scarcity, to test whether the embodied-simulation account generalizes or is Einstein-specific. - illogicalThe paper concedes that an AI could plausibly identify the conceptual inconsistency between Newtonian absolute time and Maxwell's constant speed of light that motivated Einstein's search in the first place -- an act of noticing which surprising Result is worth explaining. Yet the paper classifies this act as merely deductive ('much like a system identifying buggy code'), while its own Peircean framework defines exactly this kind of selection (choosing which Case merits explanation) as characteristically abductive. This blurs the sharp induction/deduction/abduction boundary the paper otherwise insists on, without acknowledging the tension.
Read next: Peirce's own writing on abduction as hypothesis-selection (not just hypothesis-generation) would clarify whether 'noticing an inconsistency worth resolving' counts as abduction under his original scheme.
Closes gaps elsewhere
- Why cultivating agency matters more than cultivating skills in the AI era | Max Schoening (Head of Product, Notion) · g4 — That gap asks for evidence on whether AI's coding-benchmark progress reflects real domain impact, naming 'progress in mathematical reasoning (AlphaProof)' as a specific reading direction. Zahavy engages AlphaProof directly and argues it demonstrates LLM mastery of Deduction only (formal derivation from given premises) -- not the Abduction/invention capability that would constitute genuine new-domain scientific progress. This complicates Schoening's 'domain progress is just applying coding principles' claim with a structural distinction rather than simply confirming or denying it; it does not address the gap's other named examples (AlphaFold, drug design), so treat as a partial close on the AlphaProof/math-reasoning strand specifically.