Presentation: Fixing the AI Infra Scale Problem by Stuffing 1M Sandboxes in a Single Server
Felipe Huici · rss · 2026-09-09 source ↗ Reader 🖥️
Agent SecurityCompute & Energy InfrastructureCustom Infrastructure Operators (Extending Platform Primitives)Infrastructure EfficiencySandboxing
The historical trade-off between isolation strength and speed/density in cloud infrastructure was never intrinsic to the hypervisor boundary itself — it was an engineering artifact distributed across the whole platform chain (proxy, controller, VMM, snapshot storage, host-OS resource limits) — and once each link is rebuilt for millisecond latency, snapshot-based stateful scale-to-zero turns the (still gold-standard) VM isolation boundary into the highest-density primitive available for bursty, mostly-idle agent and sandbox workloads.
practical · conference talk transcript (QCon London) with audience Q&A, delivered as a product/technology walkthrough by a vendor founder — Felipe Huici is CEO/co-founder of Unikraft; the entire talk is a technical sales pitch for Unikraft's platform, structured as history (research lineage: ClickOS, ''My VM is Lighter than your Container'', the Unikraft OSS project) plus a live demo. Vendor authorship should discount confidence on every performance figure: no independent benchmarks are cited, all numbers are Unikraft-produced, and one graph (the netboot spike) is admitted in Q&A to be an artifact of a field engineer's demo-deadline batching choice rather than a real property of the system.
Structure
| Part | Role |
|---|---|
| Framing quiz (cold boot speed, VMs-per-server, best isolation) | Sets up the counter-intuitive thesis via audience guesses before revealing the answer |
| Isolation primitives tour (VMs, microVMs, unikernels, containers, language runtimes, and their combinations) plus the trusted-computing-base (TCB) argument | Establishes the theoretical criterion (TCB size) for ranking isolation primitives, concluding containers and language isolates enlarge the TCB while the hypervisor keeps it minimal |
| Personal/research history (ClickOS, 'My VM is Lighter than your Container', Unikraft OSS project) | Narrative credibility-building: demonstrates the speed claims have a decade of prior research behind them |
| Key requirements + the AWS AMI failure story (10ms locally vs. 30s on AWS) | Reframes the problem: a fast VM image is necessary but not sufficient; the whole request path (load balancer, proxy, controller, VMM) must be re-engineered |
| Snapshot mechanism (memory-image capture/restore) and its five uses (masking startup latency, templating, stateful scale-to-zero, forking, checkpointing, migration) | Supplies the specific technical mechanism that converts millisecond-boot VMs into stateful, resumable, sleep-able units |
| Scaling to 1M: host-OS breakage stories (TAP devices, bridge port limits, kernel lock contention, NTP/SSH/tailscale failures) and their fixes | Shows the ceiling at high density was the Linux host's per-VM resource model, not the hypervisor or VMM |
| Use cases (headless browsers, functions, build/test environments, agent sandboxes) | Generalizes from the VM mechanism to workload classes sharing an idle-heavy usage pattern |
| Live demo (nginx scale-to-zero; OpenClaw agent sandboxes waking/sleeping; 1,024,132 scaled-to-zero VM instance on one box) | Empirical demonstration (vendor-controlled) of the claimed density and latency |
| Kubernetes integration via a custom virtual kubelet that lies about VM state | Shows how the millisecond-scale mechanism is made compatible with an ecosystem (K8s) that doesn't natively operate at that timescale |
| Takeaway + Q&A | States the closing claim explicitly and lets the audience probe the caveats (capacity limits, credential exfiltration, snapshot bandwidth, scheduling) |
Key terms
- microVM
A virtual machine launched by a VMM designed for fast boot (Firecracker) — identical to any VM in every other respect; not a distinct isolation category - unikernel
A VM (can itself be a microVM) whose guest is a custom, application-specific OS containing only the code the one application needs — distinguished by what's inside, not how it's launched - trusted computing base (TCB)
The layer of code shared by, and therefore critical to the isolation of, all co-located tenants; used as the criterion for ranking VMs > containers > language runtimes by isolation strength - sandbox (in this talk) Sandboxing
A general-purpose VM with full access, in which arbitrary agent/user code can run; distinguished from earlier 'sandbox' usage as a container primitive - stateful scale-to-zero
Suspending an idle VM to (near) zero CPU/memory consumption via a memory snapshot, then resuming it from exactly where it left off, indistinguishably fast to the end user - differential/compressed snapshot
A memory-image capture that stores only deltas since the prior snapshot, compressed and deduplicated (skipping freed memory), needed to make snapshot storage and restore-bandwidth tractable at six-figure-plus VM counts
Propositions
- explicitVirtual machines (hypervisor-isolated) have a smaller trusted computing base, and therefore stronger multi-tenant isolation, than containers (shared host OS/kernel) or language-level runtime isolates (shared runtime plus host OS), because the TCB is whatever code is common to all co-located tenants.conf: highsupportsHow OpenAI Built a Secure Windows Sandbox for Codex Agents · p7 — Both argue OS/hypervisor-level isolation, not application-level allowlisting or LLM-level guardrails, is the correct layer for agent containment — this doc supplies the hypervisor-level version of the same claim Codex's design makes at the OS-primitive level.
- explicitMicroVM and unikernel are not new isolation categories: a microVM is just a VM launched by a fast-booting VMM (Firecracker), and a unikernel is just a VM whose guest OS is stripped to the one application's needs; the industry vocabulary around 'isolation soup' obscures that there are really only three underlying models (VM, container, runtime isolate).conf: highsupportsGoogle Announces GKE Agent Sandbox and Hypercluster at Next '26 · p6 — A well-funded, research-pedigreed non-hyperscaler vendor building a competing microVM-based sandbox platform is itself evidence that agent code-execution sandboxing has become a competitive infrastructure category in its own right.
- explicitA fast VM image alone is insufficient for millisecond end-to-end latency; the entire request path — load balancer, proxy, VM-lifecycle controller, and VMM — must independently be engineered for millisecond behavior, as demonstrated by the AWS AMI test that produced a 30-second cold start against a 10ms local baseline.conf: high
- explicitFirecracker was chosen over QEMU (too slow) and over Intel Cloud Hypervisor (CPU jitter and idle-CPU consumption at thousands-of-VMs scale) specifically because of measured scheduling/jitter behavior at scale, not on paper feature comparisons.conf: medium
- explicitdevelopedMemory snapshotting is a single mechanism that unlocks five distinct capabilities: masking slow application startup (CI/CD-time warmup), templating (spawning many VMs from one snapshot), stateful scale-to-zero (sleep/resume with no lost state), forking (spawning sub-agent copies), checkpointing/rewind, and live migration.conf: highextends ←Google Open-Sources AX a Kubernetes Style Orchestrator for Autonomous AI Agents · p4 — AX generalizes the VM-snapshot-based 'stateful scale-to-zero' pattern to an actor/agent-session abstraction with a declarative Kubernetes-style control plane on top.
- explicitdevelopedAt six-figure-to-million VM counts, the binding constraints were the Linux host's per-VM resource model — one TAP device per VM overwhelming the kernel, Linux bridge port-count limits, kernel lock contention, and NTP/SSH/tailscale breaking — not the hypervisor, VMM, or unikernel technology itself.conf: highextendsMulti Agent Collaboration Gets Persistent Compute in Bedrock AgentCore · p4 — Bedrock AgentCore scopes microVMs as the better default specifically for short, bursty request-response agents; this talk's stateful snapshot-based scale-to-zero removes session length as the discriminator, since even long-idle sessions resume with full state in milliseconds.extends ←Beyond Kubernetes at Modal: How to Scale 1 Million Concurrent Sandboxes in Seconds · p2 — Identifies a complementary bottleneck at the same order of magnitude: where Unikraft hit host-kernel-level limits (TAP devices, bridge ports) on a single box, Modal hit cluster-coordination-level limits (etcd, centralized scheduling) across many boxes — together mapping two distinct bottleneck layers of million-instance sandbox scale.
- explicitdevelopedThe headline '1 million' figure is 1,024,132 VMs suspended (scaled to zero) on a single box, consuming near-zero CPU/memory each and a few KB of controller metadata; it is not 1 million concurrently active VMs, and the presenter concedes elsewhere that once concurrent demand exceeds available cores (e.g. 48), requests queue and boot times degrade.conf: highextends ←Beyond Kubernetes at Modal: How to Scale 1 Million Concurrent Sandboxes in Seconds · p9 — Where Unikraft/Fly's headline '1 million' figure turned out to be idle, scaled-to-zero VMs on a single box, Modal's reported 1M-sandbox benchmark claims genuine concurrent creation with sub-second startup across tens of thousands of nodes — a different, distributed-horizontal path to the same headline number, and the same scrutiny about what 'concurrent' actually measures should apply.
- explicitcorroboratedThe right cloud-capacity dimensioning question shifts, under this architecture, from 'provision for peak total users' to 'provision for peak concurrent active users,' because idle users cost near-zero when scaled to zero.conf: mediumsupports ←Google Open-Sources AX a Kubernetes Style Orchestrator for Autonomous AI Agents · p3 — Both treat an idle-heavy agent workload shape as the central cost problem infrastructure must solve, though AX applies suspend/resume at the actor level rather than the VM level.
- explicitThe Kubernetes integration works by having a custom virtual kubelet report every scaled-to-zero pod as 'running' at all times, regardless of its actual suspended state, on the premise that if wake-and-reply happens within milliseconds, the control plane's false belief is inconsequential to the end user.conf: high
- explicitSnapshot restore is described as fully transparent to the guest application ('Firecracker doesn't know or care what's inside the VM'), with no mention of state that resists transparent resume — e.g. wall-clock-dependent logic, live TCP/TLS sessions, leases/timeouts, or cached DNS/credentials that may have expired during suspension.conf: medium
- explicitCredential security for agent sandboxes should be achieved by keeping API keys and credentials outside the agent entirely, in a host-side proxy that injects them per-request under firewall rules, rather than granting the agent direct access to secrets.conf: highsupportsRunning Python code in a sandbox with MicroPython and WASM · p8 — The credential-proxy answer illustrates exactly Willison's point that sandboxing bounds blast radius but doesn't prevent agents from misusing sanctioned power (e.g. a leaked credential) within the sandbox — hence keeping credentials outside the sandbox entirely.
- explicitVM-level isolation does not by itself prevent an agent from acquiring root access inside its own guest kernel via a Linux user/kernel-space escape; it only guarantees that such an escape is contained to that guest's own kernel rather than spreading to co-tenants.conf: high
Arguments
Virtual machines (specifically, minimal hypervisor-only VMs) are the best available multi-tenant isolation primitive for cloud and agent workloads. strength: medium
Explicit premises:
- Isolation strength correlates inversely with trusted-computing-base size.
- The VM's TCB is just the hypervisor; the container's TCB is the full host OS/kernel; the language-runtime's TCB is the host OS plus the entire language runtime.
- Nesting containers inside VMs reduces to the VM's TCB anyway, since the hypervisor remains the only shared layer.
Implicit premises:
- Lines of code is a valid proxy for exploitable attack surface (no CVE data offered).
- The hypervisor itself is assumed simple/minimal enough to actually realize a small TCB — true for Firecracker's design goals, but not verified in the talk.
- Isolation strength, not performance or operational simplicity, is being held as the dominant criterion in this argument (performance is addressed separately, as a solvable engineering problem).
Inferences:
- TCB size ranking (VM < container < runtime isolate) generalizes into a security ranking without further evidence
A cloud platform can now deliver strong isolation, millisecond speed, and massive scale simultaneously, rather than picking two of three as was previously assumed. strength: medium
Explicit premises:
- VM isolation is already the strongest primitive (see prior argument).
- Every component in the request path was individually re-engineered for millisecond latency (proxy, controller, VMM, shared-memory-based inter-component comms).
- Snapshotting removes application startup latency and enables stateful scale-to-zero, which in turn enables extreme density by suspending idle VMs.
- Host-OS-level scaling bottlenecks (TAP devices, bridge limits, lock contention) were identified and specifically engineered around.
Implicit premises:
- The demonstrated 1M-VM density is representative of production conditions, not a one-off, vendor-controlled demo built for a conference deadline (the netboot-graph spike admission suggests otherwise for at least one measurement).
- Idleness (the 1,000-users/10-20-active pattern) is the dominant shape of the target workloads (agent sandboxes, headless browsers); the argument's density payoff collapses if workloads are instead frequently or continuously active.
- No independent, third-party benchmark validates any of the latency or density figures — all are self-reported by the vendor whose business depends on these claims being true.
Inferences:
- From 'we solved each sub-problem' to 'therefore the three-way trade-off is dissolved for all workloads of this shape', without addressing workloads that don't fit the idle-heavy assumption
Narratives
- The AWS AMI test: a Dockerfile-derived VM boots in 10ms locally, but the same image launched via the AWS console takes ~30 seconds.
→ Demonstrates, via a concrete failure, that a fast VM image is not sufficient for a fast platform — motivating the rebuild of the entire request-handling chain - Research lineage: ClickOS (packet-processing VMs, published as 'ClickOS and the Art of NFV'), 'My VM is Lighter (and Safer) than your Container' (8,000 VMs on a server, 28,000 citations), then the founding of the Unikraft open-source project via the Xen incubator, culminating in a second paper with 8,000 citations.
→ Establishes a decade-plus research pedigree behind the speed/density/isolation claims, lending them credibility beyond a single vendor demo - Live demo: nginx scaling to zero and back on request; OpenClaw agent sandboxes (numbered instances) waking on query ('is it sunny in London', 'what's the best UK food'), replying, and returning to sleep; a script waking all agents in sequence producing a 'snake effect'; finally, instance 1,024,132 (and 1,111,111) shown running nginx on a single box with over a million scaled-to-zero VMs present.
→ Provides live, if vendor-controlled, empirical support for the millisecond-latency and million-VM-density claims - Debugging the netboot-time graph spike (Q&A, Participant 4): the presenter admits he doesn't know the root cause and speculates a field engineer batched VM starts more aggressively (100 to 500 at once) under demo-deadline pressure to load 1 million VMs in time.
→ Inadvertently undercuts the rigor of the presented performance graphs — reveals that at least one measurement artifact traces to operational haste rather than a system property
Questions addressed
- ✅ Which isolation primitive gives the best multi-tenant security in the cloud?
VMs (via the trusted-computing-base argument), with containers and language-runtime isolation ranked progressively weaker - ✅ Can strong isolation, millisecond latency, and massive density be achieved simultaneously, or must two be traded off against the third?
Yes, per the presenter, achieved through end-to-end platform re-engineering plus snapshot-based stateful scale-to-zero — though only demonstrated on the vendor's own infrastructure with self-reported numbers - ✅ How does a millisecond-scale, sleep/wake VM platform integrate with Kubernetes, which was not designed for that timescale?
A custom virtual kubelet intercepts K8s's pod-status queries and always reports 'running,' hiding the sleep/wake cycle from the control plane - ✅ How is agent-sandbox credential/secret exfiltration prevented?
Keep credentials outside the agent, in a host-side proxy with firewall rules, injected per-request - ⭕ What happens to application state and correctness guarantees (timers, live connections, cached credentials, DNS) across a snapshot suspend/resume cycle?
- ⭕ Is it safe for a platform to systematically misreport VM/pod state to an orchestrator (Kubernetes) that other parts of the same stack rely on for accurate state (e.g. HA, self-healing, autoscaling decisions)?
- ⭕ How do the density and latency numbers hold up under independent benchmarking or third-party red-team/security evaluation, rather than vendor self-report and vendor demo?
Gaps
- incompleteNo independent benchmarks are offered for any of the headline figures (10ms cold boot, 1,024,132 suspended VMs, 12GB controller RAM, ~12TB snapshot storage projection). All numbers are self-reported by Unikraft's own CEO on Unikraft's own hardware, and one graph (the netboot-time spike) is admitted in Q&A to be a demo-deadline artifact rather than a measured system property.
Read next: Cross-reference with any third-party benchmark of Firecracker snapshot restore latency or microVM density at scale, if one exists, to check whether these figures are reproducible outside Unikraft's own stack. - incompleteSnapshot-restore transparency is asserted ('Firecracker doesn't know or care what's inside the VM') but never examined for the classes of application state that plausibly do NOT survive a suspend/resume cycle correctly: wall-clock-dependent logic, live TCP/TLS sessions and their peer state, lease/timeout-based protocols, and cached DNS resolutions or short-lived credentials that may have expired during suspension. The presenter is asked directly about transparency and answers only about the REST API surface, not the semantic correctness of resumed application state.
Read next: Look for practitioner or academic writing on the semantics of VM/container checkpoint-restore for network-stateful services (e.g. CRIU literature) to see what actually breaks. - illogicalThe virtual kubelet always reports pod status as 'running' regardless of actual suspend state, on the premise that sub-perception latency makes the lie harmless. But Kubernetes's own HA and self-healing guarantees — which the presenter explicitly cites as reasons people value Kubernetes — depend on accurate liveness/readiness signals; a control plane fed a false world-model cannot correctly diagnose a genuinely stuck or crashed VM versus one that is merely asleep, undermining exactly the properties (self-healing, HA) the talk claims to preserve.
Read next: Consider how this tension is handled (or not) in Unikraft's own documentation or in AWS/Azure's competing persistent-agent-runtime products (see extract-multi-agent-collaboration-bedrock-agentcore), which appear to expose real state rather than fabricate it. - uninformedThe entire density argument (1,000 users, 10-20 active, therefore massive overcommit is safe) rests on an assumed workload shape — that AI agent/sandbox usage is dominated by idle time — which is never measured or sourced. Agent workloads doing long-running autonomous tasks, background monitoring, or continuous tool-use loops would not fit this idle-heavy profile, and the talk does not address how the architecture degrades (beyond 'requests queue') as the active fraction rises.
Read next: Look for usage-pattern data on AI coding/browsing agent sessions (active vs. idle time distributions) to test whether the idle-heavy assumption generalizes beyond chat-style Q&A demos. - incompleteThe talk never names or engages with gVisor-based sandboxing (e.g. Google's GKE Agent Sandbox), the leading alternative agent-isolation approach that shares the host kernel via syscall interception rather than a hypervisor boundary. Given the presenter's own TCB argument, a direct comparison (hypervisor-only TCB vs. gVisor's user-space kernel-emulation TCB) would have been the natural next step, but is absent.
Read next: Read extract-gke-agent-sandbox-hypercluster alongside this doc for the missing comparative framing; look for independent security research directly comparing gVisor and Firecracker attack surfaces.
Closes gaps elsewhere
- Google Announces GKE Agent Sandbox and Hypercluster at Next '26 · g3 — Partially closes the gap. This talk supplies the missing comparative framework the GKE piece lacked — a trusted-computing-base criterion for ranking VM/container/runtime isolation, plus explicit engineering reasoning for choosing Firecracker over QEMU and Intel Cloud Hypervisor. It does NOT close the gap fully: gVisor is never named or compared, and no empirical (red-team/escape-attempt) evaluation is offered for either approach — the comparison stays theoretical (TCB size) rather than measured.
Related documents
- Netflix Moves Toward Open Source Flink Autoscaler for 30,000+ Streaming Jobs
- A big week for AI denialism
- Anthropic Details How It Contains Claude Across Web, Code, and Cowork
- The Claude Code active attack didn't stop. 294,842 secrets stolen from 6,943 machines. It evolved and now spreads through Python too and uses Claude Code itself to steal your secrets. The risk to your credentials just got bigger.
- Fragments: May 5
- Presentation: From Fab To Token - The State Of The Market
- How GitHub Is Securing Agentic Workflows in Modern CI CD Systems
- GitLab Warns That AI Agent Sandboxes Are Only as Secure as Their Network Access
- Google Announces GKE Agent Sandbox and Hypercluster at Next '26
- GKE Security Blueprint Joins Growing List of Cloud AI Frameworks