Data Mesh & Data Products
Zhamak Dehghani's domain-oriented data architecture — data organized as owned, API-driven "data products" (encapsulating data + metadata + access, with a sense/transform/quality-check/promote lifecycle) rather than centralized warehouses/lakes — the pattern autonomous data products extend to containers for AI/agent consumption.
Also known as: data mesh, data product, data products, autonomous data products, Data 3.0, domain-oriented data, data product owner, output port, data contract, domain-driven data ownership, DPROD
How authors use it
Oldest first (extraction date when no publication date is known).
| Author | Their term | Their meaning |
|---|---|---|
| Presentation: Autonomous Data Products for the Autonomous Era: Rethinking Data Architecture for GenAI Jörg Schad · 2026-07-24 | output port | One of several standardized, always-in-sync consumption formats (SQL table, file, vector embedding, MCP endpoint) generated from a single underlying data product's semantic model, so different consumer types (data scientist, ML engineer, autonomous agent) can each use their preferred modality without separate parallel pipelines. |
Documents (3)
- Enterprise Analytics Beyond Dashboards: Intelligent Data Orchestration with LLMs Nitesh Khapekar · 2026-09-14 Reader 🖥️
Enterprise analytics should stop trying to consolidate all data into ever-larger warehouses and instead be organized around an LLM-powered orchestration layer that dynamically decomposes a natural-language business question into subqueries against purpose-built, heterogeneous data stores, executes them in parallel under deterministic, pre-retrieval security enforcement, and synthesizes the results into a single auditable, source-attributed answer that no individual store could produce alone. - Forsvaret med egen data-grunnlov Silje Rognsvåg · 2026-08-18 Reader 🖥️
The Norwegian Armed Forces are building a 'data constitution' (FDSA) that makes data — not the technology or vendor processing it — the stable core of digital infrastructure, so that suppliers can be swapped without disruption; Norway's Digitalization Minister argues this same data-centric logic should extend to the rest of the public sector. - Presentation: Autonomous Data Products for the Autonomous Era: Rethinking Data Architecture for GenAI Jörg Schad · 2026-07-24 source ↗ Reader 🖥️
To make enterprise data safely and scalably consumable by autonomous AI agents, organizations must re-architect their data as domain-owned, containerized "autonomous data products" — self-contained units with a standardized lifecycle, multi-modal output ports, and centrally-enforced but independently-authored policies — rather than continuing to bolt point solutions onto today's fragmented, storage-centric "data hairball."
Propositions touching this concept (8)
By publication month, oldest first (extraction month when no publication date is known).
2026
- explicitThe histogram (i.e., simple distributional access to data) remains the highest-leverage data product; exotic tooling is overrated relative to access.
2026-07
- explicitcorroboratedAn 'autonomous data product' — a running process with a sense -> transform -> quality-check -> promote lifecycle, encapsulating data and metadata behind one interface — is the correct unit of abstraction for making data safely consumable by both humans and agents, analogous to how containers standardized the microservice world.
- explicitcorroboratedSafety and compliance policies (e.g. 'no PII exposed to an LLM') should be defined once as declarative, reusable contracts and enforced automatically at deploy-time and run-time — an 'admission controller' for data products — rather than requiring a human governance reviewer to approve each data product individually.
- explicitdevelopedThe same underlying semantic data product should be exposed through multiple always-in-sync output modalities (SQL, file, vector embedding, MCP) rather than building separate parallel pipelines per consumer type, which eliminates the 'representations falling out of sync' failure mode.
- explicitProgressive tool discovery — starting an agent with a single discovery tool and expanding the visible tool/resource set only as the agent declares intent, via MCP's subscribable list_tools endpoint — is necessary to keep autonomous consumers from being overloaded by the full tool surface of a data mesh.
- implicitcorroboratedDomain teams, not a central platform/governance team, should own the semantic definition and business logic of their data products, while a shared, centrally-run orchestration layer (the mesh/gateway) handles cross-cutting standardization, discovery and enforcement.
- implicitOnce every data asset is built as a data product, organization-wide lineage is obtained 'for free' through the abstraction itself, without a separate lineage-tracking system.
- explicitdevelopedA vault can progressively enrich (prose -> typed fields -> linked meaning -> graph projection) and, exposed as DPROD data products, scale unchanged from a private second brain to team memory to enterprise integration to an open decentralised agentic web.
Open gaps (7)
- incompleteThe claim that lineage 'comes for free' once every data asset is a data product elides that lineage across data products built on heterogeneous underlying engines (Spark, Databricks SQL, an Iceberg lakehouse) still requires each engine's own job/run metadata to be captured and correctly correlated across products — the talk asserts this correlation is trivial rather than demonstrating it, and no failure case or edge case (e.g. a data product whose transform spans two engines) is discussed.Presentation: Autonomous Data Products for the Autonomous Era: Rethinking Data Architecture for GenAI — read next: Dedicated data-lineage literature (e.g. OpenLineage) for what cross-engine lineage correlation actually requires.
- incompleteAll evidence for the architecture is either analogy (Kubernetes/Docker/LLVM), an internally-consistent lifecycle description, or one small live demo of the speaker's own vendor product (Nextdata) — no production adoption numbers, latency/overhead measurements of the four-stage lifecycle versus a raw pipeline, or independent case studies are presented, and the speaker's commercial stake as the vendor's VP Engineering is never disclosed as a limitation on the evidence (though he does gesture that 'this is one implementation' of a supposedly vendor-neutral pattern).Presentation: Autonomous Data Products for the Autonomous Era: Rethinking Data Architecture for GenAI — read next: extract-mcp-api-sprawl for hard adoption/security statistics in the adjacent MCP-governance space, as a model of what evidence-backed claims in this domain look like.
- incompleteThe article never states the geopolitical/beredskap driver explicitly — NATO and NSM involvement gesture at it, but the reader is left to infer whether this is driven by the war in Ukraine, concern about foreign (especially US) tech-vendor dependency, cloud-sovereignty debates, or purely internal efficiency goals. This is a sharper version of the same gap the corpus flagged for a finance-sector parallel (beredskap and geopolitisk lock-in as explicit drivers of exit-strategy work).Forsvaret med egen data-grunnlov — read next: Read NSM/PST/E-tjenesten public risk assessments on tech-vendor dependency in critical infrastructure, or follow-up Kode24/Digital Norway coverage of the October 5 leverandørkonferanse, for the strategic rationale FDSA's public unveiling doesn't spell out.
- incompleteNo cost or engineering-effort comparison is given between building and maintaining this five-store-plus-orchestrator architecture versus a traditional consolidated warehouse. The core claim that this is a net-better trade-off is asserted architecturally, never quantified.Enterprise Analytics Beyond Dashboards: Intelligent Data Orchestration with LLMs — read next: Production case studies or TCO benchmarks comparing federated multi-store agentic analytics architectures against consolidated-warehouse approaches over multi-year horizons.
- incompleteThe article names 'inconsistent business definitions across stores' and 'conflicting metrics with no reconciliation logic' as real production failure modes, but never assigns ownership or process for reconciling cross-store semantic drift (e.g., who decides what 'active customer' means, and how disagreement is caught before an agent silently synthesizes an answer from two conflicting definitions).Enterprise Analytics Beyond Dashboards: Intelligent Data Orchestration with LLMs — read next: Governance/ownership models for semantic consistency across independently-maintained data stores — e.g. data contracts or shared business-glossary enforcement mechanisms.
- uninformedThe article doesn't engage with prior architectural patterns aimed at the same goal — federated query engines, data virtualization layers, and text-to-SQL/NL2SQL systems predate LLM orchestration and share the 'don't move the data' principle. No comparison is offered for why the LLM-based approach succeeds where those plateaued, beyond 'GenAI removes the last-mile translation problem,' leaving the reader unable to assess how much of the claimed improvement is genuinely new versus a rebranding of federated-query architecture with a natural-language front end.Enterprise Analytics Beyond Dashboards: Intelligent Data Orchestration with LLMs — read next: Literature or vendor comparisons of pre-LLM data-federation and data-virtualization architectures against this LLM-orchestration pattern.
- incompletePhase 5 ('Result synthesis') says the LLM 'resolves contradictions' between stores, but no mechanism, confidence signal, or fallback behavior (e.g. surfacing the disagreement to the user versus silently picking a winner) is described for what happens when, say, the metrics store and a document genuinely disagree.Enterprise Analytics Beyond Dashboards: Intelligent Data Orchestration with LLMs — read next: Design patterns for surfacing (rather than silently resolving) cross-source contradictions in multi-store synthesis.