Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI

No Priors: AI, Machine Learning, Tech, & Startups · video · 2026-03-20 Reader 🖥️

Agent Context Files (CLAUDE.md / AGENTS.md)Coding Agents Emotional Impact of AI Work Jagged Capability (On Rails / Off Rails)Jevons Paradox LLM Limitations Long-Running & Managed Agents Orchestration Tax / Human Review Bottleneck Recursive Self-Improvement Riding Model Improvements Token Throughput

The jump in code-agent capability since December 2024 has made human throughput — not model capability — the binding constraint, and the central challenge is now removing yourself from the loop so autonomous agents can maximize token throughput on your behalf.

practical · mixed — Primarily practical — Karpathy describes working methods, tools, and projects — but with theoretical implications about intelligence, speciation of models, recursive self-improvement, and the future of digital vs physical economies. The form is mixed: argumentative (claims about where AI is heading), narrative (anecdotes about Dobby, auto research), and expository (explaining how code agents and auto research work).

Structure

Part	Role
The capability jump and AI psychosis	Sets up the premise: a phase change happened in December where coding shifted from 80/20 human/agent to 20/80 or beyond
Parallelization and Peter Steinberg's workflow	Illustrates the new paradigm: macro actions, multiple agents tiling a monitor, maximizing token throughput
Personality, memory, and Claude/claw design	Explores what makes agent interaction effective: personality, sycophancy calibration, memory, persistence
Dobby the home-automation claw	Narrative proof-of-concept: agents unifying six apps via API discovery, replacing bespoke software
Auto research and recursive self-improvement	Central technical contribution: describes autonomous hyperparameter optimization that beat his own expert tuning
Open auto research and untrusted compute swarms	Extends auto research to a distributed model: untrusted workers submit commits, cheap verification, blockchain-like structure
Jobs, demand elasticity, and Jevons paradox	Addresses societal implications: software demand may increase as cost drops; digital professions change first
Open source vs frontier labs	Argues open source is ~6-8 months behind and this is a healthy dynamic; draws Linux analogy
Digital vs physical: the atoms lag	Predicts digital unhobbling first, then sensor/actuator interfaces, then full physical-world automation
Education and micro GPT	Coda: the teacher's role shifts to injecting the few bits agents cannot generate; agents handle explanation and personalization

Key terms

AI psychosis Emotional Impact of AI Work
A persistent, anxious state of trying to keep up with the rapidly expanding frontier of what is possible with AI agents, feeling that any idle moment is wasted potential
skill issue
When something fails with an agent, the problem is the human operator's inability to prompt, configure, or orchestrate correctly — not a fundamental capability limitation
macro actions Coding Agents
High-level units of work delegated to agents — entire features or functionalities rather than lines or functions of code
claw Long-Running & Managed Agents
A persistent, autonomous agent system (like Claude) that loops, has memory, runs in its own sandbox, and acts on your behalf even when you are not watching
token throughput Token Throughput
The rate at which you can consume AI inference tokens — the new equivalent of GPU utilization, representing your productive capacity
auto research Recursive Self-Improvement
An autonomous loop where agents run experiments, evaluate against objective metrics, and iterate without human involvement — demonstrated by optimizing LLM hyperparameters overnight
program MD Agent Context Files (CLAUDE.md / AGENTS.md)
Markdown instruction files that describe how an auto researcher or agent should operate — essentially the 'source code' of a research organization
jaggedness Jagged Capability (On Rails / Off Rails)
The uneven capability profile of LLMs: brilliant PhD-level performance in some domains coexisting with 10-year-old-level failures in others, far more extreme than in humans
unhobbling Jevons Paradox
The release of pent-up demand and efficiency gains as AI removes bottlenecks in digital information processing
speciation Riding Model Improvements
The expected divergence of AI models into specialized variants optimized for specific domains, analogous to biological speciation
on rails / off rails Jagged Capability (On Rails / Off Rails)
Whether a task falls within the verifiable, RL-optimized domains where models excel (on rails) or outside them where performance degrades (off rails)

Propositions

explicitSince December 2024, software engineers' default workflow has completely changed from writing code to delegating to agents.conf: high
explicitcontestedWhen agent tasks fail, the problem is almost always a skill issue — the human operator's failure to configure or prompt correctly — not a fundamental capability gap.conf: high
contradictsAgents don't know what good looks like. And that's exactly the problem. · p2 — Karpathy attributes agent failures to operator skill, while the essay holds the inability to distinguish valid from invalid solutions is an unfixable design constraint.
contradicts ←Harness engineering for coding agent users · p12 — If harnesses can only partially externalize human implicit knowledge, agent failures are not reducible to operator skill issues.
explicitcontestedThe binding constraint on productivity has shifted from compute/typing speed to the human's ability to maximize token throughput.conf: high
contradicts ←Is AI coding on average 17.3x faster or 1.3x faster? Both. — LinkedIn post by Fabrice Bernhard · p3 — The binding constraint is not token throughput but downstream value-stream bottlenecks: 17.3x faster code yields only 1.3x more releases.
explicitcontestedTo get the most out of current AI tools, you must remove yourself as the bottleneck and arrange things to be completely autonomous.conf: high
refines ←[FULL WORKSHOP] AI Coding For Real Engineers - Matt Pocock, AI Hero · p9 — Narrows the full-autonomy prescription: implementation can be AFK, but alignment requires human presence and resists parallelization.
contradicts ←LLMs Corrupt Your Documents When You Delegate · p8 — Benchmark evidence that autonomous delegation silently corrupts documents cuts against the advice to arrange work to be completely autonomous.
explicitAuto research overnight found hyperparameter improvements that Karpathy, with two decades of experience, had missed in his already well-tuned model.conf: high
explicitBespoke consumer apps should not exist — everything should be exposed API endpoints with agents serving as the intelligent glue.conf: high
explicitThe customer of software is increasingly the agent acting on behalf of humans, not the human directly.conf: high
explicitcorroboratedLLM improvement via RL is limited to verifiable domains; non-verifiable capabilities like humor, nuance, and knowing when to ask clarifying questions remain stagnant.conf: high
supports ←LLMs Can't Jump · p7 — Zahavy's account of deduction (verifiable domains, e.g. AlphaProof) versus abduction (no error signal available in Einstein's case) gives a structural/epistemological explanation for Karpathy's empirical observation that RL-driven LLM gains are limited to verifiable domains and don't generalize to non-verifiable ones.
explicitIntelligence gains in verifiable domains (code) do not generalize satisfyingly to non-verifiable domains (jokes, nuance).conf: high
explicitcorroboratedWe should expect speciation of AI models — smaller specialized models with a cognitive core — rather than a single monolithic oracle.conf: medium
supports ←How the 4 New AI Models Change How You Work · p2 — The wave's throughline — cheap specialized models (voice front-ends, implementation coders) working under a smarter orchestrator — is the concrete arrival of Karpathy's predicted model speciation: smaller specialized models with a cognitive core rather than one monolithic oracle.
explicitDigital transformation will move at the speed of light compared to physical-world automation because bits are a million times easier than atoms.conf: high
explicitOpen-source models being 6-8 months behind frontier closed models is a healthy and desirable dynamic, analogous to Linux vs proprietary operating systems.conf: high
explicitCentralization of frontier intelligence in a few labs carries systemic risk; centralization has a very poor track record historically.conf: high
explicitA research organization is fundamentally a set of markdown files describing roles and connections — and this code can itself be optimized.conf: medium
explicitA distributed swarm of untrusted agents on the internet could potentially outperform frontier labs at research because the earth's total compute dwarfs any single lab.conf: medium
explicitestablishedJevons paradox will apply to software engineering: as AI makes software cheaper, demand will increase, likely creating more software engineering jobs in the near term.conf: medium
supports ←Presentation: The Free-Lunch Guide to Idea Circularity · p7 — Independent statement of the same Jevons-Paradox-for-developer-jobs claim, via different historical evidence (COBOL/low-code vs. contemporary agentic coding) -- though this corpus already flags the underlying disanalogy (task-level vs. occupation-level automation) as unaddressed in both.
supports ←Presentation: The Future of Engineering: Mindsets That Matter When Code Isn't Enough · p1 — Both independently apply Jevons paradox to conclude software-engineering demand/jobs will grow as code gets cheaper — Greene from lived startup pressure, Karpathy from labor-market reasoning.
explicitEducation is shifting from teaching humans directly to teaching agents, who then personalize explanation for each human.conf: high
explicitcorroboratedBeing inside a frontier lab compromises your independence — there are things you cannot say and organizational pressures that constrain free judgment.conf: high
supports ←What is the harder human-capital problem beneath token capital? · p5 — Karpathy: being inside a frontier lab compromises your independence — there are things you cannot say. This essay generalizes and supports that — corporate structure selects against and 'domesticates' precisely the irreverent, permission-averse talent that moves the frontier.
explicitBeing outside frontier labs causes your judgment about AI capabilities to drift because you lose access to what's coming down the line.conf: high
implicitThe agent personality — feeling like a teammate, calibrated praise, excitement — meaningfully affects productivity and user experience.conf: medium
implicitThe science of fine-tuning models without losing capabilities is insufficiently developed, which inhibits model speciation.conf: medium

Arguments

The human operator, not the AI model, is now the binding constraint on productivity. strength: high

Explicit premises:

Agent capability jumped dramatically in December 2024
Engineers shifted from 80/20 to 20/80 human/agent code writing
Token throughput is the new measure of productive capacity
When things fail, it feels like a skill issue rather than a capability gap

Implicit premises:

The models are sufficiently capable that most failures are attributable to poor orchestration rather than fundamental limits
Human cognitive bandwidth for task decomposition and review is finite

Inferences:

Mastery of agent orchestration is the new core engineering skill
Parallelizing agent work across multiple sessions is the path to maximizing output

Auto research can outperform expert human researchers in domains with objective metrics. strength: high

Explicit premises:

Karpathy had two decades of experience and had already well-tuned his model
An overnight auto research run found improvements he missed (weight decay on value embeddings, Adam beta tuning)
Hyperparameters interact jointly, making exhaustive human search impractical

Implicit premises:

The search space of hyperparameter combinations is too large for human intuition to cover
Compute cost for overnight runs is acceptable relative to the gains

Inferences:

Removing the human from the optimization loop yields better results
Frontier labs should be and likely are automating their research loops similarly

Bespoke consumer apps will be replaced by API endpoints with agent intelligence as the interface layer. strength: medium

Explicit premises:

Karpathy replaced six home automation apps with a single agent (Dobby) talking to APIs
The agent discovered, reverse-engineered, and unified disparate smart home systems in a few prompts
Agents can compose cross-app workflows that no individual app supports

Implicit premises:

API access will become more standardized and available
Users will trust agents with control over their physical environment
Security and authentication challenges can be solved

Inferences:

The app economy will restructure around API-first, agent-consumed interfaces
Software UX as a discipline shifts from designing for humans to designing for agent consumption

RL-trained models improve only in verifiable domains, creating persistent jaggedness rather than general intelligence gains. strength: medium

Explicit premises:

The 'atoms' joke has persisted unchanged for years despite massive model improvements
Models are trained via reinforcement learning on verifiable outcomes
Softer capabilities (humor, nuance, knowing when to ask) are not easily verifiable

Implicit premises:

If general intelligence transfer were happening, improvements in code would bleed into humor/creativity
The labs have not prioritized or found ways to create rewards for non-verifiable capabilities

Inferences:

The 'smarter at code implies smarter at everything' thesis is not holding
Closing the jaggedness gap requires new training paradigms beyond current RL

Open-source models ~6-8 months behind frontier is a healthy and sustainable dynamic for the industry. strength: medium

Explicit premises:

Linux is extremely successful despite proprietary alternatives
Industry needs a common open platform for safety and trust
Current open-source models are good enough for vast majority of consumer use cases
Centralization has a poor historical track record

Implicit premises:

The capex gap between frontier labs and open-source efforts will not widen indefinitely
The value of frontier intelligence is concentrated in rare, high-stakes use cases

Inferences:

Open source eating basic use cases while frontier labs push boundaries is an acceptable division
More frontier labs (not fewer) would improve the ecosystem

Digital transformation will dramatically outpace physical-world automation, with interfaces between digital and physical being the next major opportunity. strength: high

Explicit premises:

Bits can be copied and manipulated millions of times faster than atoms
Self-driving — the first robotics application — required enormous capital and time, and most startups failed
There is a digital overhang: insufficient human thinking cycles have been applied to already-digital information

Implicit premises:

The patterns observed in self-driving will repeat in other physical-domain applications
The total addressable market for physical automation is actually larger, just slower to materialize

Inferences:

Near-term focus should be digital; mid-term on sensor/actuator interfaces; long-term on full physical automation
Companies building at the digital-physical interface (sensors, lab equipment, data collection) represent the next wave

Narratives

Karpathy built 'Dobby the elf claw' — a home automation agent that discovered his Sonos system via IP scan, reverse-engineered the API with web searches, played music in three prompts, then extended to lights, HVAC, shades, pool, spa, and a security camera system using a Quinn vision model for change detection, all controlled via WhatsApp.
→ Demonstrates the thesis that agents can unify disparate software systems through API discovery and natural language, replacing six dedicated apps with a single conversational interface.
Peter Steinberg sits in front of a monitor tiled with Codex agents, each running ~20 minute tasks across 10+ repos checked out simultaneously, moving between them giving work and reviewing results.
→ Exemplifies the new paradigm of parallelized agent orchestration and macro-level software manipulation — the workflow Karpathy aspires to master.
Karpathy ran auto research overnight on his already-tuned NanoChat GPT-2 model and it discovered missed optimizations: weight decay on value embeddings and insufficiently tuned Adam betas — improvements his two decades of experience had not found.
→ Provides concrete evidence that autonomous agent loops can exceed expert human performance in domains with objective metrics.
Karpathy went around OpenAI telling colleagues that if they succeed, they are all out of a job — they are building automation for the CEO/board and are essentially glorified auto researchers automating themselves away.
→ Illustrates the existential tension of working at a frontier lab: the researchers actively building the technology that replaces them.
Karpathy tried to get an agent to write micro GPT — to boil down neural network training to its simplest form — but the agent could not do it. His obsessive simplification to 200 lines is his unique value-add; the agent perfectly understands and explains it but cannot originate it.
→ Defines the boundary of current AI capability: agents excel at explanation and implementation but cannot match human taste and obsessive simplification.
The 'atoms' joke test: asking ChatGPT to tell a joke returns 'Why do scientists not trust atoms? Because they make everything up' — the same joke from years ago, unchanged despite massive model improvements.
→ Vivid illustration of model jaggedness and the limits of RL training: capabilities outside the verified reward signal remain frozen.

Questions addressed

✅ What has changed in software engineering practice since the code-agent capability jump?
Engineers have shifted from writing code to orchestrating multiple parallel agents, operating in macro actions over repositories rather than line-by-line coding. The human is now the bottleneck.
✅ Can autonomous AI research loops outperform expert human researchers?
Yes, in domains with objective metrics. Auto research overnight beat Karpathy's two decades of tuning experience on hyperparameters.
⭕ Will AI replace software engineering jobs?
Cautiously optimistic: Jevons paradox suggests demand for software will increase as cost drops. Long-term is uncertain. Karpathy defers to economists for proper forecasting.
✅ Do intelligence gains in verifiable domains generalize to non-verifiable ones?
Not satisfyingly. The persistent 'atoms' joke and jaggedness of models suggest RL improvements in code do not transfer meaningfully to humor, nuance, or social intelligence.
⭕ Should AI models be a monoculture or should we expect speciation?
Karpathy expects speciation but acknowledges the science of fine-tuning without capability loss is immature. Current market dynamics favor monoculture because labs serve unknown end users.
✅ Is open-source AI viable and sustainable alongside frontier closed models?
Yes — currently 6-8 months behind, covering most consumer use cases. This gap is healthy and analogous to Linux's role alongside proprietary OSes.
✅ Why is Karpathy not at a frontier lab?
Financial incentive alignment compromises independence; inside a lab you cannot freely speak or act. Outside, your judgment drifts. The ideal may be going back and forth.
✅ What will happen to physical-world automation relative to digital?
Digital will transform first (bits are a million times easier), then digital-physical interfaces (sensors/actuators), then full physical automation. The physical TAM is larger but slower.
✅ How will education change with AI agents?
Teachers shift from explaining directly to curating the few bits of insight agents cannot generate. Agents handle personalized explanation. Documentation becomes markdown for agents, not HTML for humans.

Gaps

incompleteThe 'skill issue' framing is never seriously challenged. The conversation does not explore cases where agent failures are genuinely caused by capability limitations rather than user error, creating an unfalsifiable narrative where every failure is attributed to the human.
Read next: Research on systematic LLM failure modes that persist regardless of prompting quality (e.g., planning over long horizons, maintaining state across complex multi-step tasks).
closed byLLMs Corrupt Your Documents When You Delegate — DELEGATE-52 documents systematic failures (25% corruption even in frontier models, unimproved by agentic tooling) that persist regardless of operator skill — the counter-evidence to the unfalsifiable skill-issue framing the gap asks for.
incompleteAuto research is demonstrated only on hyperparameter optimization — a narrow, well-defined search with clear metrics. No evidence or discussion of how it would work on research requiring genuine novelty, taste, or paradigm shifts.
Read next: Literature on AI-driven scientific discovery beyond optimization: can agents formulate novel hypotheses, not just test existing ones? See work on AI scientists (e.g., Sakana AI's 'The AI Scientist').
uninformedThe Jevons paradox argument for software engineering jobs is applied without discussing the significant disanalogies: ATMs automated a narrow task within banking, while AI agents potentially automate the entirety of software production. Demand elasticity analysis is hand-waved.
Read next: Economic analysis of AI labor displacement that distinguishes between task-level automation (Jevons applies) and occupation-level automation (potentially different dynamics). See Acemoglu vs Brynjolfsson debate.
incompleteSecurity and trust implications of the 'apps dissolve into APIs' vision are acknowledged but not seriously explored. Dobby discovered unprotected Sonos APIs — this is a security vulnerability presented as a feature.
Read next: Research on agent-to-API security models, OAuth for agents, and the attack surface of agent-mediated home automation.
incompleteThe conversation between hosts and Karpathy is largely uncritical — hosts rarely push back on claims, and potential downsides of the depicted future (concentration of power, loss of human skill, surveillance via home agents) are not explored.
Read next: Critical perspectives on AI agent dependency: skill atrophy in engineering, principal-agent problems when delegating to opaque systems, and power dynamics of the 'agent economy'.
closed by16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden — Provides the critical perspective on agent dependency the gap seeks: learning requires effortful practice, so agent delegation erodes junior developers' tacit knowledge and destroys the judgment needed to delegate well.
illogicalKarpathy simultaneously claims (1) everything is a skill issue and current tools are sufficient, and (2) models are jagged, burst at the seams, and going too far ahead makes them net negative. These are in tension — if the tools truly suffice, jaggedness should be manageable.
Read next: Framework for distinguishing 'skill issues' from genuine capability boundaries in AI agent systems.

Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI

Structure

Key terms

Propositions

Arguments

Narratives

Questions addressed

Gaps

Related documents