Presentation: Parting the Clouds: The Rise of Disaggregated Systems

Murat Demirbas · rss · 2026-07-30 source ↗ Reader 🖥️

theoretical · mixed — Primarily theoretical/expository — a survey of how and why database architecture has shifted to disaggregation — but built on an argumentative core (the Paxos-role analogy) and narrated through historical case studies (Aurora, PolarDB, TaurusDB) and forward-looking speculation (future work). Practical implications (which mitigations to apply, when to disaggregate) are secondary to the explanatory thesis.

Structure

Key terms

• Disaggregation
  Architecturally separating compute and storage into independently-scaling components connected only via the network, replacing a design where both live in the same process/box.
• Impedance mismatch
  The structural mismatch between compute (costly, fast-fluctuating demand, can be stateless) and storage (cheap, stable/growing demand, inherently stateful), which makes bundling them together economically wasteful.
• Log is the database Compute-Storage Disaggregation
  Aurora's design principle: ship only the write-ahead/redo log to storage (not the full materialized state), and have storage nodes reconstruct pages from the log — cutting network traffic and data movement.
• Log Store / Page Store
  TaurusDB's further split of the storage tier itself into a low-latency, durability-prioritized Log Store and a separate Page Store that materializes pages for buffer/read serving.
• Proposer / Acceptor / Learner (mapped to compute / log store / page store)
  Lamport's three Paxos roles, which the author maps onto the disaggregated architecture: proposers generate order (compute), acceptors persist votes/the log (log store, durability), learners serve materialized state (page store, availability).
• Ingrained fault tolerance
  The design principle that consensus/coordination should be built into the disaggregated database service itself, rather than delegated to an external configurator or coordination service that designates a primary node.

Propositions

• explicitThe differing cost and scaling profiles of compute (costly, elastic, statelessness-compatible) and storage (cheap, stable/growing, inherently stateful) make bundling them in one box economically wasteful and violate separation of concerns.conf: high
  extendsPodcast: Increasing Users' Data Agency: From BlueSky's AT Protocol to the Local-First Software Movement · p1 — Kleppmann states as a passing premise that cloud-native systems decouple storage from compute; Demirbas supplies the underlying economic mechanism (cost/scaling/statefulness mismatch) and concrete production examples (Aurora, PolarDB, TaurusDB) behind that observation.
  extendsDesigning Data-Intensive Applications: The Cloud & Doing the Right Thing · p3 — ddia-cloud asserts generally that cloud-native architecture outperforms lift-and-shift deployment; Demirbas gives the specific architectural mechanism (compute/storage disaggregation) that produces that performance gap.
• explicitDisaggregating compute from storage enables independent elastic scaling — including scaling compute to zero — which is what enables pay-per-use pricing for customers.conf: high
• explicitDisaggregation provides fault isolation — a node crash loses only compute or only storage, never both — enabling fast recovery and simplified operations.conf: high
• explicitThe first wave of 'cloud-ified' databases (Raft/Paxos-replicated monolithic boxes, combined with cloud storage that is itself already replicated) did not actually solve disaggregation and is economically wasteful — e.g., three-way Raft replication on top of cloud storage that is already replicated three times yields nine-times storage replication.conf: high
• explicitAurora's 'log is the database' innovation — shipping only the redo log to storage and letting storage materialize pages from it — is the shared foundational invention that subsequent disaggregated database architectures (PolarDB, TaurusDB, Socrates) generally built on or varied.conf: high
• explicitLamport's original description of Paxos in terms of three distinct roles — proposer, acceptor, learner — is structurally identical to today's disaggregated database architecture's split into compute, log store, and page store.conf: high
• explicitMulti-Paxos and Raft compacted Paxos's three roles into a single node for a shared-nothing architecture, which is why the field lost sight of the resemblance between classical consensus roles and today's disaggregated systems.conf: high
• explicitcorroboratedBecause acceptors need only 2f+1 for quorum/no-split-brain while proposers and learners need only f+1, replicating uniformly across all roles (as Raft effectively does once roles are merged) is more redundant than the classical Paxos role decomposition requires — and decades of Paxos flexible-quorum research can be reused to improve disaggregated database throughput.conf: high
  supports ←Presentation: How To Run on Three Clouds at Once, and When Not To · p7 — Demirbas's abstract point that Raft-style replication needs a quorum majority (2f+1-style reasoning) to avoid split-brain is directly instantiated by Form3's concrete case: a nine-pod Raft cluster needing 5 of 9 alive, and the cross-cluster PDB gap that could have violated that quorum.
• explicitExisting disaggregated database architectures typically depend on an external configurator/consensus service to designate the primary node — a violation of 'ingrained fault tolerance' (the principle that a system should be self-contained rather than depend on external services).conf: high
• explicitDisaggregation shifts the systems bottleneck from CPU/disk to the network: remote I/O is roughly three times slower in latency and four times lower in bandwidth than local SSDs.conf: high
• explicitA shared, disaggregated storage layer makes it easier to unify previously separate specialized systems (e.g., OLTP and OLAP in Google AlloyDB) on top of the same storage, because both can now read the same fresh data.conf: medium
• explicitTrue serverless databases require disaggregation as a precondition, because stateless compute workers that can appear and vanish safely depend on shared durable storage for fast failover and cold start.conf: high
• explicitDisaggregation, despite improving fault isolation for single-component failures, weakens 'fate-sharing' between compute and storage and introduces new failure surfaces (e.g., metastable failures) from increased component count and information lag between layers.conf: high
• explicitNew AI/agentic workloads will produce traffic bursts roughly thirty times worse than historical patterns, which is a key motivation for pursuing self-assembling, auto-tuning database systems as future work.conf: medium
• implicitThe persuasive force of the Paxos-to-disaggregation analogy rests on role-count and role-function correspondence (three roles, similar functions) rather than on any demonstrated correctness-preserving mapping between Paxos's quorum-based safety guarantees and the actual, independently-engineered consistency mechanisms of commercial systems (Aurora's ad hoc 4-of-6 quorum, PolarDB's parallel Raft).conf: medium

Arguments

Narratives

• The three-phase history of database architecture: monolithic (Postgres/MySQL since the 1970s) -> cloud-ified monolith replicated via Raft/Paxos groups with two-phase commit for horizontal scale -> genuinely disaggregated architecture (Aurora onward).
  → Frames disaggregation as the current endpoint of a linear, inevitable historical progression rather than one design choice among several, reinforcing the unity's claim that cloud economics 'demands' this direction.
• The step-by-step walk through Aurora ('log is the database', storage-side materialization surgery on MySQL/Postgres), PolarDB (compute-side InnoDB materialization, parallel Raft, RDMA via PolarFS), and TaurusDB (Log Store / Page Store split, custom scatter replication).
  → Supports the claim that 'log is the database' is the shared invention while showing real engineering variation, letting the audience see the pattern as convergent rather than copied.
• Lamport's own remark in the Multi-Paxos paper — that the three roles can be compacted into one node 'if needed for efficiency' — used to explain how the field came to forget that Paxos was originally a disaggregated, three-role design, only to reinvent that decomposition decades later as disaggregated databases.
  → Supports the central Paxos analogy with a 'history rhymes / nothing new under the sun' framing that makes the connection feel discovered rather than constructed.
• The self-correcting 'I lied to you a little bit' moment: after presenting fault isolation as an unqualified benefit earlier in the talk, the author narratively reverses to explain that disaggregation weakens fate-sharing and opens the door to metastable failures.
  → A staged rhetorical reveal that makes the downside land harder than if it had been presented flatly alongside the benefits earlier — supports the proposition that disaggregation's fault-isolation benefit is real but incomplete.

Questions addressed

• ✅ How does disaggregated database architecture relate to classical consensus protocols like Paxos?
  Resolved via the role-decomposition analogy: proposer/acceptor/learner map onto compute/log-store/page-store, and Multi-Paxos/Raft's compaction of those roles explains why the resemblance was lost.
• ✅ What mitigates the network latency and bandwidth costs that disaggregation introduces?
  Buffering, prefetching, pipelining, faster fabrics (RDMA, CXL), custom protocol restructuring (e.g., overlapping two-phase-commit phases), log-traffic prioritization, scatter I/O, and pushdown compute to storage.
• ⭕ Can database systems become self-assembling or auto-tuning to handle the burstier traffic patterns of agentic AI workloads?
  Explicitly flagged as an open question and future-work direction, not yet solved.
• ⭕ Is the vendor-adoption trend cited as proof that disaggregation 'is already here' (Aurora 2015 through Amazon DSQL 2025) actually evidence that these systems deliver the claimed economic and performance benefits at scale, or merely evidence of adoption?

Gaps

• incompleteThe talk's central analogy (Paxos's proposer/acceptor/learner = disaggregated compute/log-store/page-store) is argued from role-count and role-function correspondence, but never examines whether the actual consistency and safety mechanisms engineered into commercial systems — Aurora's specific 4-of-6 storage quorum, PolarDB's 'parallel Raft' — were derived from, or are even formally equivalent to, Paxos's quorum-based correctness guarantees. It is left unclear whether this is a rigorous technical lineage or a structural resemblance noticed after the fact and narrated as history.
  Read next: Compare the Aurora and PolarDB architecture papers' own consistency arguments against Lamport's Paxos correctness proof, to see whether their quorum choices were derived from consensus theory or arrived at independently by systems engineers optimizing empirically for latency and durability.
• incompleteThe claim that 'decades of Paxos research... can be borrowed' to improve disaggregated database throughput is illustrated only by the author's own separate research prototype (compartmentalized Paxos, an 8x throughput result over the same codebase) — not by evidence that any of the three surveyed commercial systems (Aurora, PolarDB, TaurusDB) actually apply flexible quorums or role-disaggregated consensus in production.
  Read next: Read the compartmentalized Paxos paper (Demirbas et al. with UC Berkeley collaborators) and any public engineering retrospectives from the Aurora/PolarDB/TaurusDB teams to check whether flexible-quorum results have moved from research into production, or remain a proposal.
• incompleteThe list of production disaggregated systems (Aurora 2015 through Amazon DSQL 2025) is presented as confirmatory evidence that disaggregation 'is already here' and validated, with no counter-examples, documented cost overruns, performance regressions, or teams that reverted from disaggregated to monolithic designs — a one-sided evidentiary base for what is framed as an economically inevitable trend.
  Read next: Look for independent benchmarks or critical retrospectives comparing Aurora/PolarDB/AlloyDB total cost of ownership and performance against self-hosted or lift-and-shift Postgres/MySQL, to test whether 'cloud economics demands disaggregation' holds outside an idealized cost model.
• illogicalThe 'ingrained fault tolerance' principle (build consensus into the database rather than depend on an external configurator) is asserted without reconciling that internalizing consensus reintroduces the coordination overhead that motivated separating log stores and page stores from compute in the first place. The author himself concedes the shared-log/virtual-log research branch he praises still depends on an external metastore ('I'll live with it'), which undercuts the principle's own claim to full self-containment without resolving the tension.
  Read next: Read Mahesh Balakrishnan's shared-log/virtual-log work for how it explicitly handles this exact tradeoff, rather than fully internalizing consensus — it may show that full self-containment is not achievable in practice and 'ingrained fault tolerance' is aspirational rather than a settled design principle.

Related documents

• Designing Data-Intensive Applications: The Cloud & Doing the Right Thing
• Podcast: Increasing Users' Data Agency: From BlueSky's AT Protocol to the Local-First Software Movement