Presentation: From Copy-Paste to Composition: Building Agents Like Real Software

Jake Mannix · presentation · 2026-07-27 source ↗ Reader 🖥️

practical · conference-talk transcript: a practitioner problem/solution pitch, structured as diagnosis (agents are built like 1975 BASIC) followed by a concrete worked proposal (virtual tools + taint tracking + registry), delivered first-person to a peer engineering audience — QCon AI presentation by a Walmart Global Tech Technical Fellow. Explicitly framed by the speaker as a working prototype ('vibe coded... over a weekend or two'), not a shipped or benchmarked system — the talk is argumentative advocacy for an architecture, not a report of production results.

Structure

Key terms

• virtual tools
  An intermediate, versioned mapping layer between an agent definition and an MCP tool server that can rename a tool, rewrite its description, project/constrain its parameter schema, and compose several tools into one — without modifying the underlying tool or the MCP protocol.
• lethal trifecta Lethal Trifecta
  Simon Willison's named risk pattern, used here as the concrete security threat the virtual-tools/taint-tracking machinery is built to defend against.
• taint tracking
  Labeling tool inputs/outputs with data-classification tags (e.g. 'returns PII', 'accesses external web') and propagating that classification through the agent's context so a policy layer can act on it deterministically.
• composition
  Combining atomic, single-purpose MCP tools into coarser-grained, purpose-built functions or workflows (sequential/parallel chaining, fan-out/fan-in, conditionals) the way ordinary programs compose functions.
• schema projection
  Exposing only a reduced sub-schema of a tool's parameters to the LLM — hiding or hard-wiring the rest (e.g. always setting an organization ID) with deterministic defaults instead of leaving them for the model to infer.
• eval pack
  A registry-stored triple of a typed eval dataset (input schema, output schema), an eval runner, and an LLM-as-judge scoring configuration — framed as an 'inductive PRD' that both engineers and non-technical stakeholders can read, clone, and modify.

Propositions

• explicitIf we're going to build LLM-based agents like real software rather than ad hoc scripts, we need actual notions of interfaces, encapsulation, versioning, and dependency management applied to agent tooling — the things regular software engineering built up over the last 50 years.conf: high
• explicitBuilding LLM-based agents is programming in a new, highly-specified, natural-language programming language — English.conf: high
• explicitCurrent agent-building practice replicates the structure of 1975-era BASIC: one big file/prompt, no interfaces, no encapsulation, and non-deterministic 'GOTO'-like control flow.conf: high
• explicitMCP was a necessary and genuinely good step forward because it factors specialized tool functionality out of the agent into separate servers owned by domain experts (Slack knows Slack, Stripe knows payments), avoiding N-teams-times-M-tools duplicated build effort.conf: high
• explicitAuto-generating MCP servers directly from an OpenAPI spec produces low-quality tools, because good tool granularity, parameter count, and description clarity require deliberate design decisions about what the LLM actually needs — decisions an automated conversion cannot make.conf: high
• explicitBecause MCP tool name/description/schema is concatenated verbatim into the agent's system prompt, agent authors have no way to rename, wrap, or override a tool's interface — they inherit the tool author's exact wording as literal, load-bearing instructions to their own LLM.conf: high
• explicitBecause MCP gives them no ability to change the contract, agent authors are accountable for eval quality while effectively letting the tool author write half of their program's controlling text.conf: high
• explicitA 'virtual tools' layer — an intermediate mapping layer sitting between the agent definition and the tool server — can rename a tool, rewrite its description, and constrain/project its parameter schema, all without modifying the underlying tool or the MCP protocol itself.conf: high
  extendsAgent Harness Engineering · p12 — Both converge on tool assembly moving from static, hand-authored configuration toward something dynamic and composable; this talk supplies a concrete mechanism (a versioned virtual-tools mapping/override layer) for the general trend the other document only predicts.
• explicitSchema projection — hiding parameters the agent doesn't need, or hard-wiring them to deterministic defaults (e.g. always the engineering Slack channel, or a side-channel user ID) instead of exposing them to the LLM — reduces context-window confusion and removes an error-prone class of LLM-inferred values.conf: high
• explicitVirtual tools give tool authors a clear, stable boundary: instead of fielding a stream of conflicting per-agent naming/wording change requests against one canonical description, they publish a default and let each agent team override it independently.conf: high
• explicitVirtual tools give agent authors ownership of the interface (naming, description, constraints) they need to tune their own evals and optimize for their own business outcomes, without waiting on any other team.conf: high
• explicitcorroboratedMCP's tool list being fully dynamic and unversioned by default is a bug, not a feature: if a tool's schema/description can change on the very next request, evals against that agent cannot be run reliably or reproduced.conf: high
  supportsFighting Tool Sprawl: The Case for AI Tool Registries · p5 — Independently arrives at the same conclusion — that a production-grade agent tool registry must include versioning as a first-class capability, not an optional extra — from a security/eval-reproducibility angle rather than a sprawl-management angle.
  supports ←The Interfaces Are Arriving · p12 — Independent evidence for treating MCP as immature, version-unstable infrastructure: a fully dynamic, unversioned tool schema breaks eval reliability the same way the article's own five-revisions-in-under-two-years cadence breaks integration stability.
• explicitThe 'lethal trifecta' — an agent context combining access to untrusted content, an external communication channel, and access to private/internal data — creates a structural data-exfiltration risk that is common in production agents, not an edge case, because agents doing meaningful work over internal data routinely have all three capabilities at once.conf: high
• explicitAdding guardrail instructions to the system prompt ('please don't exfiltrate data') is not a security strategy, because LLMs are probabilistic, can be wrong on any given call, and are routinely jailbroken by publicly circulated adversarial inputs.conf: high
• explicitDeterministic taint tracking — labeling tool outputs/inputs with data-classification tags (e.g. 'returns PII', 'accesses external web', 'takes user input') and propagating that classification through the context — lets a policy layer make enforcement decisions (e.g. disallow external-web tools once PII is present) without depending on the model's own judgment.conf: high
  extendsIndirect Prompt Injection Exploits GitHub's AI Agent to Leak Private Repository Data · p8 — The other document argues prompt injection resists the SQL-injection fix because there is no clean channel separation between instructions and data in current LLM systems; this talk offers taint-classification tracking through the context as a substitute systematic defense that does not require achieving that channel separation.
• explicitTaint-based policy can and should be enforced at three separate points: static lint/analysis at tool build or agent registration time (reject or flag disallowed capability combinations before deployment), a human-approval gate for flagged-but-not-outright-disallowed combinations, and dynamic runtime enforcement.conf: high
• explicitThe best runtime enforcement does not block a tool call mid-execution (which confuses the agent and produces a visible failure) but proactively shrinks the visible tool set — e.g. by pushing an MCP 'tools/list_changed' notification once PII is detected in context, so the next planning step never even sees the exfiltration-capable tools.conf: high
  extendsPrompt Injection to Data Exfil in 3 Hops · p10 — The other document argues that removing the exfiltration channel is the one actionable leg of the lethal trifecta since it can only be 'contained deterministically'; this talk supplies a concrete containment mechanism — taint-triggered, proactive MCP tool-list revocation before the next planning step — for exactly that containment.
• explicitThe virtual-tools and taint-tracking scheme can be implemented as an intermediary proxy/gateway that speaks MCP on both sides (agent-facing and server-facing) — a Strangler Fig pattern — without changing the MCP specification, which matters because MCP is now governed by the Linux Foundation and widely deployed, making direct spec changes slow and high-friction.conf: high
  supportsAWS MCP Server Reaches GA with Full API Coverage and IAM-Based Governance · p2 — Both independently converge on interposing a single governed intermediary (an MCP proxy/gateway in this talk, a single IAM-audited MCP server in the other) between agent and tool servers as the practical lever for both governance and security, rather than changing agent code or the protocol itself.
• explicitVirtual tools, policies, and taint rules need to live in a persistent, versioned registry so that agent definitions can pin exact dependency versions (e.g. 'notify-engineering v1.3'), and proxies cache them locally instead of querying the registry on every request.conf: high
• explicitA tool/agent registry should store typed input and output schemas — not bare strings — so tool outputs can be consumed programmatically, and so dependency graphs among agents, tools, and services can be built and analyzed for blast-radius impact when something changes.conf: high
• explicitEvals should be stored in the registry as an 'eval pack' — a typed dataset (input schema, output schema), an eval runner, and an LLM-as-judge scoring config — functioning as an inductive, example-based product-requirements-document that non-data-scientist business people can author and read.conf: high
• explicitOnce evals live in a shared registry with typed schemas and a runner already pointed at the agent, other stakeholders (security, finance, ML teams) can run adversarial or economic experiments against an agent's published eval suite without meetings — e.g. red-teaming with translated prompt-injection payloads, or benchmarking a cheaper or fine-tuned model for a Pareto-optimal cost/quality swap.conf: high
• explicitThe proposed virtual-tools-plus-taint-policy approach sits at the lightweight, config-driven end of a spectrum whose far end is full workflow engines (BPMN, Temporal) or letting the LLM write and execute its own code directly (CodeAct); the speaker deliberately starts at the lightweight end because it does not require ripping out an existing MCP-based agent stack.conf: high
• explicitThe speaker has prototyped ('vibe coded') the virtual-tools and taint-tracking mechanisms and confirms they work functionally, but has not validated them for performance or scale — an explicit maturity caveat on the whole proposal.conf: high
• implicitThe talk implicitly treats software-engineering discipline (interfaces, encapsulation, versioning) as cleanly transferable to a system whose reasoning core remains fundamentally probabilistic — i.e. that layering deterministic structure around a non-deterministic model is sufficient to get most of the benefits of real software engineering, even though the model's own planning and tool-use decisions inside that structure remain unpredictable.conf: medium

Arguments

Narratives

• The speaker opens by polling the room: who has run an agent in production, who has written their own MCP tool (not just used someone else's server), and who has connected those tools to a production agent.
  → establishes shared practitioner credibility with the QCon AI audience and frames the talk as peer engineering discourse, not a vendor pitch, before the critique of MCP begins.
• Asked to illustrate 'composition,' an image model (Nano Banana) produces an unrequested, impossible Escher-Lego sculpture; the speaker keeps it, delighted, then admits that's apparently what 'composition' means to an LLM.
  → a comic, self-deprecating aside that marks the gap between what was asked for and what a generative model actually produces, used as a rhetorical pivot into defining 'composition' precisely and technically for the rest of the talk.
• At the close, the speaker admits the whole virtual-tools/taint-tracking system was 'vibe coded' over a weekend or two as an experiment, working functionally but untested for performance or scale.
  → a calibration/honesty narrative that tempers the pitch's ambition, distinguishing 'this architecture is sound and buildable' from 'this is production-proven,' and implicitly inviting the audience to build the real version.

Questions addressed

• ✅ How do you build LLM-based agents like real software rather than one-off, unstructured scripts?
  Insert a versioned 'virtual tools' mapping layer between agent and MCP tool server (giving interfaces, encapsulation, composition) and back it with a shared registry storing tools, policies, taint rules, typed schemas, dependency graphs, and eval packs.
• ✅ How do you retrofit interfaces, encapsulation, and versioning onto MCP tool-calling without rewriting the MCP specification itself?
  Implement the virtual-tools layer as an MCP-speaking proxy/gateway using the Strangler Fig pattern — it intercepts and rewrites tool lists on both sides without touching the underlying protocol or deployed servers.
• ✅ How do you prevent a lethal-trifecta agent context from exfiltrating data, given that you cannot rely on the LLM to follow safety instructions under adversarial pressure?
  Deterministic taint tracking: label tool inputs/outputs with data-classification tags, propagate the classification through context, and enforce policy at build/registration time, via human review gates, and via runtime dynamic tool-list mutation (rather than mid-call blocking).
• ✅ Should an MCP agent's tool list be allowed to change dynamically at any time, or should tools be versioned and pinned?
  Tools should be versioned and pinned via the registry (e.g. 'notify-engineering v1.3'); fully dynamic, unversioned tool lists are characterized as a bug that makes reliable evals impossible.
• ⭕ Who is authorized to clone and run an adversarial eval pack (e.g. a red-team prompt-injection test) against another team's live production agent, and does that itself need to pass through the talk's own capability-gating policy?

Gaps

• incompleteThe talk explicitly flags but does not resolve what happens to an agent's in-flight plan when the runtime taint-tracking layer removes tools mid-session: the speaker acknowledges the agent 'is going to think... I'm going to do this thing first... Then the tools are going to disappear. Then you're like, I'm going to do something different now. I don't know what I'm going to do,' and concedes 'it's not perfect' — but offers no mechanism for graceful replanning, partial-completion handling, or user-facing explanation when this occurs.
  Read next: Read alongside work on agent replanning and mid-task tool-availability changes, and against extract-turing-post-5-patterns-long-running-agents on checkpoint/resume patterns for long-running agents, to see whether any existing pattern handles a shrinking tool surface mid-plan without silent agent confusion.
• incompleteThe proposal makes the virtual-tools/policy/taint-rule registry a single, central, load-bearing store that every agent proxy depends on ('you're not making calls to the registry at runtime' softens this, but the source of truth is still one registry) — yet the talk never addresses who governs write access when multiple teams want to override the same underlying tool, how version conflicts are resolved, or the operational risk of the registry itself becoming a single point of failure or chokepoint for every agent in the organization.
  Read next: Read alongside extract-aws-mcp-server-ga#g3 (single-vendor MCP chokepoint risk) and extract-fighting-tool-sprawl-ai-registries#g4 (registries don't fix the organizational incentives that produce sprawl) — both flag the same class of governance gap this talk's registry proposal inherits without addressing.
• uninformedThe taint-tracking scheme is presented as a simple, one-way labeling mechanism ('this tool returns PII data... the whole context is now considered PII') without engaging with the mature static/dynamic taint-tracking and information-flow-control literature it borrows its name from — in particular, sanitizers, declassification, and taint-propagation-through-joins. The talk never explains whether taint can ever be legitimately cleared once applied, or how a tool that has a genuine business need to process PII (e.g. a support-ticket tool) would be distinguished from one that shouldn't.
  Read next: Read alongside extract-github-codeql-declarative-security#g1, which raises the identical open question — what security customization needs remain outside the sanitizer/validator pattern in taint-based models — for a different (static-analysis) domain; comparing the two would show whether agent-context taint tracking needs its own theory of sanitization or can borrow one wholesale.
• illogicalThe talk's eval-registry pitch (any team can clone and run an adversarial eval pack — including deliberately injected exfiltration prompts — against another team's live production agent 'without talking to you... no meetings are needed') sits in tension with its own defense-in-depth security argument earlier in the same talk, which insists that access to sensitive tool-capability combinations must be gated at build/registration time or reviewed by a human. Running an adversarial red-team clone against a live production agent is exactly the kind of action the talk's own taint/registration-gate logic would seem to require gating, yet the eval-registry section presents it as a frictionless, ungated self-service capability.
  Read next: Re-read propositions p13-p17 (deterministic gating of risky tool/capability combinations) against p21-p22 (frictionless self-service eval cloning and execution) to see whether the speaker's own framework, applied consistently, would require an approval gate on who may execute an adversarial eval pack against a production agent.

Closes gaps elsewhere

• Indirect Prompt Injection Exploits GitHub's AI Agent to Leak Private Repository Data · g2 — That gap flags the absence of any engagement with model-level or architectural defenses against prompt injection — specifically naming 'provenance/taint-tracking of untrusted content' as a missing angle. This talk supplies exactly that: a concrete architectural mechanism (label tool inputs/outputs with data-classification tags, propagate the classification through the agent's context, and enforce policy via build-time linting, registration-time human review, and a runtime MCP tools/list_changed notification that proactively removes exfiltration-capable tools once taint is detected) that pushes back on the claim that prompt injection can only be managed by permission scoping and content isolation.

Related documents

• AWS MCP Server Reaches GA with Full API Coverage and IAM-Based Governance
• Agent Harness Engineering
• Prompt Injection to Data Exfil in 3 Hops
  Agent Security Deterministic Egress Containment Lethal Trifecta MCP (Model Context Protocol)
• Anthropic Details How It Contains Claude Across Web, Code, and Cowork
  Agent Security Deterministic Egress Containment Lethal Trifecta
• AWS Releases Loom, an Open-Source Reference Platform for Governing AI Agents at Enterprise Scale
  Agent Security Agent Tool Management MCP (Model Context Protocol)
• Why AI agents need a Context Lake
  Agent Security Agent Tool Management MCP (Model Context Protocol)
• GitLost: How We Tricked GitHub's AI Agent into Leaking Private Repos
  Agent Security Deterministic Egress Containment Lethal Trifecta
• 11) AI-spesial med Vilde Opsal og Anders Sveen
  Agent Security Evals Software Design Fundamentals
• Article: Securing MCP in Production: Defense-in-Depth Beyond the Gateway
  Agent Security Deterministic Egress Containment MCP (Model Context Protocol)
• Vibe Coding Agent Security And Evaluation
  Agent Security Evals MCP (Model Context Protocol)