Toward a Causal Data Management Ecosystem for Decision Making and Agentic AI

The paper argues that modern AI has shifted from shipping a model to operating an ecosystem where classical ML predictors, deep learning models, large language models, retrieval pipelines, and agents run side by side, each trained over heterogeneous sources—tables, graphs, documents, images, audio streams, logs, support tickets, code repositories, and business applications—and each producing outputs that become inputs to the others. The authors contend that "operating such an ecosystem is fundamentally a data integration problem—the knowledge it depends on is fragmented across dozens of heterogeneous, independently governed sources that must be reconciled and continually maintained."

However, they argue that "integration alone is not enough. Many interacting factors shape the outputs AI systems produce, but not all of these factors carry equal weight: some are true drivers of an outcome, while others merely accompany it without causing it. Current data management systems primarily exploit correlations and statistical dependencies in the observed data, while causal relationships, interventions, and counterfactuals are not represented as first-class system primitives. This becomes critical when agents act autonomously: as agents change the very systems they observe, prediction is no longer sufficient: to be trustworthy and reliable, an agent must anticipate the consequences of its own actions—what its next action will set in motion, and what would have followed from a different one rather than extrapolate from what has co-occurred before."

The authors therefore propose treating cause and effect as shared ecosystem infrastructure, which they call the Causal World System (CWS): a persistent, explicit, queryable causal model that overlays the architectures, systems, and processes of an organization, a lab, or the broader AI ecosystem. The CWS is composed of a structural causal model (SCM) whose variables are the meaningful quantities of the organization—prices, inventory, latencies, ticket volumes, sentiment, churn, experiment arms—and whose mechanisms encode how interventions on some of these propagate to the rest. Critically, it is not a static analytics dashboard but an artifact that humans, agents, and learning systems query and interact with continuously.

Architecture and the mediated causal schema. Building on classical data integration, the CWS lifts the Global-As-View (GAV) / Local-As-View (LAV) duality from data to causal structure: sources publish local views, a mediator... reconciles them into a global SCM, and causal queries are answered against the mediated schema by rewriting them over the views. The architecture proceeds through an integration layer that handles data cleaning, transformation, and modeling (representing entities, relations, and events), followed by causal discovery (using methods such as PC, GES, FCI, and NOTEARS), causal inference (estimating effects and answering what-if questions), and counterfactual reasoning (exploring alternative worlds and outcomes). A "defining commitment of the CWS is that this structure is white-box: it is an explicit causal DAG over named ecosystem variables, not a latent representation. Every variable, edge, and estimation is inspectable and carries its provenance, so an answer is accompanied by the mechanisms and assumptions it rests on: making the CWS auditable and interpretable by construction. The system is also multimodal by construction: a return spike (tabular), its explanation in tickets (text), and the product photo that misled buyers (image) are one causal story told in three modalities," requiring alignment onto shared causal variables.

Consumers of the CWS. The paper describes three classes of consumers. Humans are served across the full query spectrum, from analytical and aggregate queries to prescriptive and explicitly causal/counterfactual ones—e.g., instead of predicting support tickets will rise next week, the CWS answers raising the price by 5% will raise tickets by 12% and churn by 3%, but discounting shipping offsets two-thirds of that. Agents are converted from a reactive predictor into a deliberative actor able to ask had I done B instead of A, would the outcome have been better?—the kind of counterfactual reasoning that prediction-only world models cannot support, and the kind that safety increasingly demands as agents gain authority to act. Models benefit from causality-powered, multi-level training, where conditioning training and fine-tuning on causal structure suppresses spurious shortcuts, improves robustness under distribution shift, and reduces the number of samples needed to reach a given level of performance. Because the CWS exposes causal structure as a hierarchy of local and global views, a model local to one source can be trained against that source's local causal view, a subsystem model against an integrated regional view, and an ecosystem-wide model against the global causal structure.

Multi-level inference. The view hierarchy enables queries at multiple granularities: a source-local view answers fine-grained mechanism questions, a subsystem view answers team-level diagnostics, and the global view answers ecosystem-level prescriptions. The formal device ensuring consistency is causal abstraction: coarse and fine views are τ-abstractions of one another, so a coarse query is answered by marginalizing over mechanisms it does not depend on, with a guarantee that the answer agrees with the fine-grained model. This lets the mediator pick the cheapest altitude that still identifies the effect while providing an audit trail that a latent world model cannot expose.

Challenges. The paper identifies five research challenges: (C1) causal discovery and integration at ecosystem scale—merging partial, possibly conflicting local causal views into a sound global one while tracking identifiability; (C2) view maintenance under drift, requiring incremental maintenance: detecting stale edges, unreliable counterfactual estimates, and LAV sources that require re-certification, analogous to materialized-view maintenance over causal mechanisms; (C3) multimodal causal alignment—mapping latent encodings from JEPA-style models onto shared causal variables with calibrated uncertainty; (C4) counterfactual reasoning as an agent primitive, which is not only an agent-interface problem, but also a database-style query-optimization problem requiring cost-aware planning, caching, indexing, and incremental or approximate evaluation; and (C5) identifiability, trust, and governance—questions such as when a query effect is identifiable, who may write to the CWS, how contested edges are adjudicated, and how counterfactuals that authorized an agent's action are audited.

The paper concludes that as AI shifts from single models to ecosystems, the binding constraint is no longer predictive accuracy but the ability to reason about consequences, and realizes that the CWS demands progress across causal discovery and fusion, view integration and maintenance, abstraction, and provenance—no single community's problem, and precisely the kind of grand challenge that the entire AI ecosystem, from learning and databases to systems and safety, is positioned to take up together.

• Causal world model as infrastructure: The AI system maintains a persistent, queryable structural causal model (SCM) over named organizational variables—prices, inventory, latencies, ticket volumes, sentiment, churn—rather than a one-off prediction model.

→ It can answer what-if and why queries directly, e.g., Raising price by 5% will raise support tickets by 12% and churn by 3%; discounting shipping offsets two-thirds of that, with full traceability of the mechanisms.

• White-box, provenance-backed causal DAG: Every variable, edge, and estimation is explicit and inspectable, carrying provenance from sources to estimates.

→ An auditor or user can inspect exactly which data sources and assumptions support a recommended action, making the system auditable and interpretable by construction.

• Causal query rewriting over mediated views (GAV/LAV for causality): The system integrates local causal views from heterogeneous sources into a global SCM using a mediator, then rewrites causal queries over those views.

→ It answers queries at the right granularity efficiently—source-local mechanism questions, team-level diagnostics, and ecosystem-level prescriptions—with guarantees of consistency across levels via τ-abstraction.

• Counterfactual reasoning as a first-class agent primitive: Agents can query the CWS for counterfactuals (Had I done B instead of A, would the outcome have been better?) and use the causal graph to plan interventions.

→ Autonomous agents become deliberative actors that anticipate the consequences of their own actions, rather than reactive predictors extrapolating from correlations—improving safety and reliability when they change the systems they observe.

• Causality-powered multi-level model training: The system exposes causal structure at multiple levels, so models are trained/fine-tuned conditioned on the relevant local or global causal view.

→ Models suppress spurious shortcuts, become more robust under distribution shift, and require fewer samples to reach a given performance level because they learn mechanisms rather than correlations.

• Multimodal causal alignment: The CWS aligns heterogeneous modalities—tabular returns, text tickets, product images—onto shared causal variables with calibrated uncertainty.

→ A single causal story can be told across modalities: a return spike in tables, its explanation in text tickets, and the misleading product image are unified, enabling cross-modal causal explanations and diagnosis.

• Incremental causal view maintenance under drift: The system detects stale edges and unreliable counterfactual estimates and re-certifies local sources, analogous to materialized-view maintenance over causal mechanisms.

→ It remains accurate over time as the underlying ecosystem evolves, without full rediscovery, and can flag when a previously valid causal conclusion is no longer trustworthy.

• Identifiability tracking and governance: The system tracks which query effects are identifiable from available data and who may write/adjudicate contested causal edges.

→ It can reject or flag queries whose effect is not identifiable, provide governance for contested edges, and audit the counterfactuals that authorized an agent's action—supporting trust and compliance.

• Cost-aware counterfactual query optimization: Counterfactual queries are treated as database-style query-optimization problems, using caching, indexing, incremental evaluation, and approximate methods.

→ The system can answer complex what-if and counterfactual questions at scale, in real time, even across large ecosystems, with a tunable trade-off between cost and accuracy.

Modern AI is no longer a single model but an ecosystem: classical ML predictors, deep and multimodal models, large language models, and agents, each trained and tuned over different data sources and each producing outputs at scale that become inputs to the others. Operating such an ecosystem is fundamentally a data integration problem - the knowledge it depends on is fragmented across dozens of heterogeneous, independently governed sources that must be reconciled and continually maintained. Yet integration alone is not enough. The predictions these systems make are shaped by many interacting factors, and the events, decisions, and variables that drive an outcome are routinely entangled with the ones that merely accompany it; treated as a basis for action, such correlational signals invite confounded decisions. This becomes acute once agents act autonomously: to be trustworthy and reliable, an agent must anticipate the consequences of its actions, not merely extrapolate from what has co-occurred before. Causal reasoning is what closes this gap, distinguishing the drivers of an outcome from its correlates, and enabling prescriptive and counterfactual analysis over the ecosystem's data. We therefore argue that the integrated ecosystem needs an explicit causal layer, and we propose to build it as a shared, persistent, queryable Causal World System (CWS).