FOD#151: Recursive Self-Learning: Why It Matters Now
Turing Post · email · 2026-05-04 Reader 🖥️
Recursive self-learning — AI systems automating parts of the process by which AI systems themselves improve — is shifting from a six-decade-old theoretical speculation to a concrete engineering reality, and the critical open question is no longer whether it will happen but how to maintain alignment and verification once the improvement loop becomes substantially autonomous.
theoretical · expository — A newsletter essay that synthesizes the concept of recursive self-learning (RSL) from historical roots (Turing, Good, Schmidhuber) through current implementations (Karpathy's autoresearch, frontier lab practices) to near-term forecasts (Clark's 2028 prediction). Primarily expository and framing — it names and contextualizes a trend rather than prescribing action.
Structure
| Part | Role |
|---|---|
| Opening frame: loops as the basic unit of ML | Establishes the familiar feedback-loop paradigm (predict, feedback, update) and names the human as the historical constant in the loop |
| Jack Clark's forecast and definition of RSL | Anchors the essay in a specific, authoritative claim — 60%+ probability of no-human-involved AI R&D by end of 2028 |
| Karpathy's autoresearch as worked example | Provides the clearest small-scale illustration: a system that edits its own training code, runs experiments, keeps improvements, discards failures |
| Historical lineage (Turing 1950 → Good 1965 → Schmidhuber 2003 → AlphaGo Zero → AutoML) | Shows RSL is not a new idea but a long-theorized one that lacked practical traction until now |
| Why AI R&D is uniquely automatable | Argues that AI research's digital-native nature (code, data, training runs, benchmarks) makes it tractable for automation unlike, e.g., biology |
| Evidence mosaic: benchmarks and partial loops connecting | Lists SWE-Bench, METR, CORE-Bench, MLE-Bench, PostTrainBench, kernel optimization as accumulating evidence |
| Closing: alignment and verification obligation | Raises but does not resolve the safety question — autonomous improvement loops need continuous verification |
Key terms
- recursive self-learning (RSL) Recursive Self-Improvement
A system automating parts of the process by which it — or systems like it — get better: writing code, generating training data, running experiments, optimizing kernels, fine-tuning models, building evaluations, improving prompts, and eventually training successor systems - the human bottleneck Orchestration Tax / Human Review Bottleneck
The observation that human researchers are now the rate-limiting factor in the ML improvement loop — they cannot manually enact every idea fast enough - autoresearch Recursive Self-Improvement
Karpathy's specific implementation: an agent given a real LLM training script that autonomously edits code, runs 5-minute experiments, measures validation bits-per-byte, keeps improvements, discards regressions, and repeats - no-human-involved AI R&D Recursive Self-Improvement
Jack Clark's framing: a system capable of training its own successor without human participation in the execution loop - child machine Recursive Self-Improvement
Turing's 1950 proposal to build a simple machine and educate it, rather than programming adult intelligence directly — the earliest articulation of the RSL idea - Gödel Machine Recursive Self-Improvement
Schmidhuber's 2003 formal framework: a system that rewrites its own code once it can prove the rewrite is an improvement
Propositions
- explicitThe human researcher is now the bottleneck in the AI improvement loopconf: high
- explicitRSL is not a model waking up and choosing to become better — it is a system automating parts of the process by which systems like it improveconf: high
- explicitAI R&D is uniquely automatable because most of the work is already digital (code, data, training runs, benchmarks, logs)conf: high
- explicitcorroboratedThere is a 60%+ probability that no-human-involved AI R&D arrives by end of 2028 (attributed to Jack Clark)conf: mediumsupports ←Does code quality still matter? · p11 — Seemann's 'Acceleration' scenario (LLMs writing think-pieces to train successor models faster than humans absorb knowledge) is a concrete illustration of the kind of no-human-involved AI R&D loop that FOD#151 estimates at 60%+ probability by 2028.
- explicitThe researcher's role shifts from manually enacting ideas to designing the loop that enacts themconf: high
- explicitRSL has moved from speculative vision to concrete systems problem with designable parametersconf: high
- explicitThe evidence for RSL is not one benchmark but an accumulation of partial loops beginning to connectconf: high
- explicitWhen a system begins evolving autonomously, it needs rigorous, continuous verification and alignmentconf: high
- explicitWe still do not really know how these machines 'think,' which makes alignment of self-improving systems fundamentally hardconf: high
- implicitFrontier labs are already working on RSL — experimenting on smaller models and removing researchers from the execution loopconf: medium
Arguments
RSL — automating the researcher's execution role — is both motivated and feasible strength: medium
Implicit premises:
- Faster AI progress is desirable; the competitive dynamics of frontier labs incentivize removing the bottleneck
RSL is becoming a practical engineering reality, not merely a theoretical possibility strength: medium
Implicit premises:
- Benchmark progress on subtasks is a reliable indicator of progress toward full-loop automation
RSL does not eliminate the researcher but changes the researcher's role from executor to loop designer strength: high
Implicit premises:
- This upward move in abstraction is a general pattern, not specific to Karpathy
Narratives
→ Provides historical legitimacy and explains why RSL is arriving now rather than earlier
→ The essay's closing frame — elevates RSL from a technical capability to a conceptual shift in the relationship between humans and AI systems
Questions addressed
- ✅ What is recursive self-learning, and how does it differ from ordinary ML feedback loops?
RSL is defined as a system automating parts of the process by which it or systems like it improve — distinct from ordinary ML loops because the target of optimization is the improvement process itself, not just the task - ✅ Why is RSL becoming practical now when the idea is six decades old?
AI R&D is digital-native (code, data, runs, benchmarks), making it uniquely tractable for automation; current model capabilities now match the requirements of R&D subtasks - ✅ What changes about the researcher's role when RSL is in place?
The researcher moves from inside the loop (manually enacting ideas) to above it (designing the loop, setting metrics, budgets, and research programs) - ✅ What evidence supports the claim that RSL is imminent?
A mosaic of benchmark results (SWE-Bench, METR, CORE-Bench, MLE-Bench, PostTrainBench), Karpathy's autoresearch demo, ICLR 2026 workshop, and $500M startup funding - ⭕ How do we maintain alignment and safety once improvement loops become autonomous?
Gaps
- The essay raises alignment and verification as 'very hard problems' but offers no concrete approaches, frameworks, or even directions for solving them — the safety discussion is entirely gestural
- Clark's 60%+ probability claim for no-human-involved AI R&D by 2028 is presented without methodological backing — it is a forecast from authority, not an argued probability
- The essay does not distinguish between automating incremental improvements (hyperparameter tuning, kernel optimization) and automating genuinely novel research directions — these are qualitatively different challenges lumped together under 'RSL'
- No discussion of failure modes, diminishing returns, or reasons RSL might plateau — the narrative is monotonically progressive
- The competitive dynamics driving RSL (lab racing incentives, geopolitical pressure) are not examined, despite being arguably the primary force accelerating deployment ahead of safety work