Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
Chenyu Zhou, Huacan Chai, Wenteng Chen, Zihan Guo, Rong Shan, Yuanyi Song, Tianyi Xu, Congming Zheng, Jiachen Zhu, Zeyu Zheng, Zhuosheng Zhang, Xingyu Lou, Zhihui Fu, Jianghao Lin, Weinan Zhang ยท article ยท 2026-04-09 Reader ๐ฅ๏ธ
Practical progress in LLM agents increasingly depends not on improving model weights but on externalizing cognitive burdens into external infrastructure โ memory, skills, protocols, and harness engineering โ which together form a systems-level framework of cognitive artifacts.
theoretical ยท expository (survey/review with argumentative framing) โ A 54-page tech report surveying agent infrastructure through a unifying theoretical lens ('externalization'). The abstract signals a taxonomy paper with an explicit conceptual argument.
Structure
| Part | Role |
|---|---|
| Observation of shift | Sets up the phenomenon: agents are built by reorganizing the runtime, not by changing weights |
| Externalization as unifying lens | Introduces the core theoretical framework drawn from cognitive artifacts |
| Taxonomy of four externalization forms | Decomposes externalization into memory (state across time), skills (procedural expertise), protocols (interaction structure), and harness (coordination/governance) |
| Historical progression | Traces evolution from weights to context to harness |
| Trade-offs and future directions | Examines parametric vs. externalized capability, self-evolving harnesses, shared infrastructure, evaluation, governance |
Key terms
- externalization Intent Debt
The process of moving cognitive capabilities out of model weights into external runtime infrastructure (memory, skills, protocols, harness), reframing hard internal problems as more tractable external ones - cognitive artifacts Intent Debt
External structures that transform cognitive burdens into forms a model can solve more reliably โ the theoretical grounding borrowed from cognitive science - harness engineering Agent Harness
The unification layer that coordinates memory, skills, and protocols into governed execution; the infrastructure that makes external modules reliable - skills (externalized) Agent Skills
Reusable procedural expertise stored outside the model and invoked at runtime, as distinct from in-weight procedural knowledge - protocols (externalized) Agent-to-Agent Protocols
Externalized interaction structure โ rules and conventions governing how agents communicate and coordinate, stored outside weights
Propositions
- explicitcorroboratedLLM agents are increasingly built by reorganizing the runtime around them rather than by changing model weights.conf: highsupports โThis Week in AI: Rethinking the Agent Harness ยท p7 โ Stanford evidence that harness design gaps outweigh model choice backs the claim that agent capability is now delivered by reorganizing the runtime, not changing weights.
- explicitcorroboratedAgent infrastructure matters because it transforms hard cognitive burdens into forms the model can solve more reliably, not merely because it adds auxiliary components.conf: highsupports โAgent Skills ยท p10 โ Concrete instance of externalization activating latent capability: the model has read Hyrum's Law in training but only applies it when a skill makes it do so.
- explicitThere is a historical progression from weights to context to harness in how agent capability is delivered.conf: high
- explicitcorroboratedMemory, skills, and protocols are three distinct but coupled forms of externalization.conf: highsupports โPrompting Claude Fable 5 ยท p12 โ The recommended Markdown notes file is a concrete instance of memory-as-externalization: transforming a cognitive burden (retaining lessons across runs) into external infrastructure the model reads, exactly the externalization thesis.refines โxAI Releases Grok Skills and Updates Tool Calling Responses API ยท p9 โ The vendor comparison qualifies the review's clean taxonomy: 'skills' is not one well-defined externalization category โ vendors mean meaningfully different things by it.
- explicitHarness engineering is not just another form of externalization but serves as the unification layer that governs the other three.conf: high
- explicitThere is a meaningful trade-off between parametric and externalized capability that must be navigated in agent design.conf: medium
- implicitThe cognitive artifacts framework from cognitive science is the appropriate theoretical lens for understanding agent infrastructure.conf: medium
- explicitSelf-evolving harnesses and shared agent infrastructure represent the frontier of externalization.conf: medium
Arguments
Practical agent progress depends on better external cognitive infrastructure, not only on stronger models. strength: medium
Explicit premises:
- Agents are increasingly built by reorganizing runtime rather than changing weights
- Capabilities previously expected to be internal are now externalized
- Externalization transforms hard cognitive burdens into more tractable forms
Implicit premises:
- Model weight improvements alone face diminishing returns or insufficient reliability for agent tasks
- The cognitive artifacts framework from human cognition validly transfers to LLM agent systems
Inferences:
- Since externalization transforms difficulty rather than merely adding components, infrastructure is architecturally central, not peripheral
Memory, skills, protocols, and harness form a coherent taxonomy of externalization. strength: medium
Explicit premises:
- Memory externalizes state across time
- Skills externalize procedural expertise
- Protocols externalize interaction structure
- Harness coordinates them into governed execution
Implicit premises:
- These four categories are exhaustive of the relevant externalization forms
- The categories are sufficiently distinct to justify separate treatment
Inferences:
- A unified framework can explain diverse agent engineering practices through a single concept
Narratives
- A historical progression narrative: from weights (training) to context (prompt engineering, RAG) to harness (full infrastructure engineering).
โ Supports the thesis by framing externalization as an inevitable evolutionary trajectory, not an ad-hoc design choice - The cognitive artifacts analogy: just as humans use external tools (writing, diagrams) to offload cognition, LLM agents use memory stores, skills, and protocols.
โ Provides theoretical legitimacy by connecting to established cognitive science, and reframes infrastructure as cognitive transformation rather than mere engineering plumbing
Questions addressed
- โ
Why is agent infrastructure increasingly important relative to model improvements?
Because infrastructure externalizes cognitive burdens into forms the model handles more reliably โ it is a cognitive transformation, not just auxiliary support. - โ
How should we categorize the different forms of agent infrastructure?
Through a four-part taxonomy: memory, skills, protocols, and harness (as unification layer). - โญ What is the right balance between parametric and externalized capability?
Identified as an open trade-off; the abstract signals discussion but not resolution. - โญ How should externalized agent systems be evaluated and governed?
Identified as an open challenge.
Gaps
- incompleteThe transfer of the 'cognitive artifacts' framework from human cognition to LLM systems is asserted but not defended. Human cognitive artifacts augment a general intelligence; LLM externalization compensates for specific architectural limitations (finite context, no persistent state). The analogy may be more misleading than illuminating.
Read next: Look for critiques of anthropomorphic framings of LLM capabilities; compare with distributed systems or compiler theory as alternative analogies. - incompleteThe taxonomy assumes memory, skills, and protocols are 'distinct but coupled' โ but the boundaries may be artificial. A skill with embedded memory and protocol-like interfaces blurs all three categories. The abstract does not address how to handle such overlap.
Read next: Check whether the full paper provides clear criteria for categorizing borderline cases. - uninformedThe abstract does not address whether externalization introduces new failure modes (brittleness, composability failures, versioning conflicts) that may offset reliability gains. The framing is almost entirely positive.
Read next: Look for systems engineering literature on integration risk; check if the full paper's 'open challenges' section addresses this. - illogicalThe historical progression 'weights -> context -> harness' is presented as a clean trajectory, but in practice these co-evolve. Larger context windows reduce the need for some externalization; better weights reduce the need for elaborate harnesses. The narrative oversimplifies.
Read next: Consider whether a co-evolutionary or dialectical framing would be more accurate than a linear progression.
Related documents
- The Anatomy of an Agent Harness
- Audit your Agent files
- How to Actually Prompt Fable 5, Straight From the Team That Built It
- Google Just Dropped a Masterclass on Agentic Engineering (It's SO Good)
- This Week in AI: Rethinking the Agent Harness
- Stop Using /init for AGENTS.md
- Paweล Huryn's Post
- Prompting Claude Fable 5
- Tony Seale's Post
- Agent Skills: Google-Backed AI Coding Agent Workflow