Agentic Engine Optimization (AEO)

Addy Osmani · rss · 2026-04-10 source ↗ Reader 🖥️

Agent-Native Design Agent Skills Coding Agents Token Efficiency

Developer documentation must be restructured for AI coding agents as a new primary consumer — optimizing for discoverability, parsability, and token efficiency via a layered stack of standards (robots.txt, llms.txt, skill.md, Markdown formatting, token surfacing) — because agents read fundamentally differently from humans and current analytics cannot detect their failures.

practical · prescriptive essay / industry analysis — Primarily a practical guide for developer documentation teams, structured as problem diagnosis followed by a layered solution stack. Draws on one empirical study but is mostly experience-driven.

Structure

Part	Role
Introduction and problem framing	Establishes that AI agents are an invisible, growing share of documentation consumers whose failures go undetected by current analytics.
How AI agents actually read documentation	Provides empirical grounding (citing the Developer Experience with AI Coding Agents paper) showing agent HTTP behavior differs radically from human browsing.
The token problem	Identifies token economics as the key technical constraint agents face, with concrete examples of oversized documents.
The AEO stack (Layers 1-6)	Core prescriptive content: a six-layer framework from access control through UX bridges, each with concrete implementation guidance.
AGENTS.md and broader implications	Extends AEO beyond documentation sites to code repositories and reflects on the paradigm shift in developer experience design.
Audit checklist and where to start	Compresses the essay into an actionable prioritized checklist.

Key terms

Agentic Engine Optimization (AEO) Agent-Native Design
The discipline of structuring, formatting, and serving technical content so AI coding agents can discover, parse, and reason over it effectively.
token economics / token efficiency Token Efficiency
The constraint imposed by LLM context windows on how much documentation an agent can consume; makes document length a functional property, not just a style concern.
llms.txt Agent-Native Design
A Markdown-formatted sitemap hosted at a domain root that provides AI agents with a structured, token-efficient index of documentation.
skill.md Agent Skills
A declarative file that tells agents what a service/API can accomplish (capabilities, constraints, required inputs) — distinct from procedural API reference docs.
capability signaling Agent-Native Design
Documentation that tells agents what an API does and can accomplish, as opposed to merely how to call it.
silent failure
When an agent fails to use documentation (due to blocking, truncation, or poor structure) and no analytics event is generated — the failure is invisible to the documentation owner.

Propositions

explicitAI coding agents are already a significant and growing share of developer documentation traffic, but their consumption is invisible to conventional client-side analytics.conf: high
explicitToken count is now a first-class documentation metric because agents use document size to decide whether to attempt reading content at all.conf: high
explicitA six-layer stack (robots.txt, llms.txt, skill.md, content formatting, token surfacing, Copy-for-AI) constitutes the necessary infrastructure for agent-optimized documentation.conf: high
explicitWhen agents cannot parse documentation effectively, they fall back to parametric knowledge (hallucination), producing subtly wrong outputs that are hard to trace back to documentation failures.conf: high
explicitOptimizing documentation for agents tends to also improve it for humans, because the disciplines (clear structure, concision, front-loaded information) overlap.conf: medium
explicitcorroboratedTeams that optimize for agent consumption early will gain a competitive advantage because their APIs will be preferentially recommended and successfully integrated by agents.conf: medium
supports ←Co-Existence and the End of Co-Intelligence · p8 — Loosely analogous, lower-confidence case: Mollick's book-site experiment (testing marketing copy against many AI models via Codex) mirrors AEO's claim of early-optimizer advantage, though AEO#p6 is specifically about APIs being preferentially recommended, not persuasive page copy.
explicitdevelopedThe traditional developer experience design paradigm (progressive disclosure, visual hierarchy, interactive examples) is partially obsolete for agent consumers.conf: high
extends ←Meta Open-Sources Astryx, Its Agent-Ready React Design System · p6 — AEO argues the human-oriented DX paradigm (progressive disclosure, visual hierarchy, interactive examples) is partially obsolete for agent consumers of documentation; Astryx applies the same logic one layer down, to a UI component library itself, replacing purely visual/prop-based customization with agent-callable CLI/MCP interfaces.
implicitDocumentation quality and agent task success are decoupling — you can have excellent docs that agents still fail to use.conf: high

Arguments

Developer documentation teams need to adopt AEO as a discipline alongside (or integrated with) traditional documentation practices. strength: high

Explicit premises:

AI agents are a significant and growing share of documentation consumers.
Agents read documentation in fundamentally different ways from humans (single requests, no client-side interaction).
Current analytics cannot detect agent consumption or failure.
When agents fail to parse docs, they hallucinate — causing subtle downstream errors.

Implicit premises:

Documentation teams should optimize for all significant consumer categories, not just the historically dominant one.
The cost of agent hallucination (wrong API usage, broken integrations) is high enough to justify documentation restructuring.
Agent traffic will continue growing, making the problem more acute over time.

Inferences:

Because agents are invisible to current analytics, teams currently have no feedback loop for agent-facing doc quality.
The absence of feedback creates a silent failure mode where doc teams believe everything is fine while agents consistently fail.

Token count should be treated as a first-class documentation metric. strength: high