CELEBI: Design and Realization of a Reproducible Physics Analysis Framework
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: High Energy Physics: Experiment. The day in high-energy experiment: detectors, searches and limits.
Kai: We're looking at the paper: "CELEBI: Design and Realization of a Reproducible Physics Analysis Framework".
Mira: Here's the gist of the paper. Reproducibility in modern physics analysis requires more than preserving source code or individual workflow descriptions<ref:2611.01788#pg3> Celebi, a framework for organizing and preserving evolving physics analyses while keeping the analysis definition lightweight and separate from generated data products<ref:2611.01788#pg6> The paper introduces Celebi, an analysis-management framework designed to provide such a representation<ref:2611.01788#pg4> It achieves this by separating the analysis definition from generated data products into an Analysis Repository and a Production Factory connected by Yuki middleware<ref:2611.01788#pg3> Framework Architecture The Celebi system separates the representation and preservation of an analysis from the execution of the corresponding computations<ref:2611.01788#pg4> Its architecture consists of two main components: the Analysis Repository and the Production Factory, connected through the Yuki middleware<ref:2611.01788#pg4> The Analysis Repository contains lightweight materials required to define an analysis, including source code, configurations, parameters, documentation, and dependency information<ref:2611.01788#pg5> The Production Factory is responsible for executing preserved analysis states and managing the resulting data products<ref:2611.01788#pg4> Yuki provides the interface between the Analysis Repository and the execution infrastructure, resolving dependency structures to construct Snakemake workflows<ref:2611.01788#pg4> This separation allows the analysis representation to remain independent of a particular execution backend<ref:2611.01788#pg4> Analysis Representation Design The design focuses on three closely related aspects: workflow architecture, file organization, and the impression mechanism<ref:2611.01788#pg5> The workflow architecture describes the analysis in terms of materials required to define each computation and their dependencies<ref:2611.01788#pg5> It distinguishes between an algorithm, which describes how a computation is performed, and a task, which describes a particular application of that computation with specified parameters and upstream dependencies<ref:2611.01788#pg5> The workflow records dependencies among the materials required to define the analysis rather than only the temporal or logical ordering of execution steps<ref:2611.01788#pg5> State Preservation Mechanism The impression mechanism preserves immutable states of objects together with their corresponding states of their dependencies<ref:2611.01788#pg5> An impression is an immutable representation of a Celebi object at a particular analysis state, recording materials defining the object along with references to impressions corresponding to its dependencies<ref:2611.01788#pg7> Each impression is assigned a persistent identifier that remains associated with that immutable state<ref:2611.01788#pg8> This mechanism allows generated results to be associated unambiguously with the analysis state from which they originated<ref:2611.01788#pg5> Repository Structure Realization The file organization realizes the workflow model in a Celebi repository using four object types: project, directory, algorithm, and task<ref:2611.01788#pg6> Project and directory objects provide the hierarchical organization of the repository<ref:2611.01788#pg6> Algorithm and task objects represent the materials that participate directly in the workflow<ref:2611.01788#pg6> Dependencies are expressed between objects rather than through hard-coded paths to generated outputs, allowing tasks to assign local aliases to upstream analysis components<ref:2611.01788#pg6> The concepts introduced provide a coherent representation of an evolving physics analysis and establish a foundation for reproducibility, provenance tracking, collaboration, and long-term analysis preservation<ref:2611.01788#pg5> The framework allows the repository to remain lightweight while retaining the information required to describe its logical structure and computational configuration<ref:2611.01788#pg6> Together, these elements provide a foundation for provenance tracking, reproducible execution, collaborative development, and long-term preservation of high-energy-physics analyses<ref:2611.01788#pg8> The detailed implementation of the execution layer is outside the scope of this paper<ref:2611.01788#pg4> The final architecture supports different execution backends without changing the underlying analysis model<ref:2611.01788#pg4> REFERENCES
1: Monya Baker.
Kai: 500 scientists lift the lid on reproducibility.
Mira: Nature, 533:452–454, 2016
2: Xiaoli Chen et al.
Kai: Open is not enough.
Title and authors: Mira: So, to frame what we’re hearing today, "CELEBI: Design and Realization of a Reproducible Physics Analysis Framework" is about building this whole system where the definition stays separate from the execution. It's not just a better way to save files; it's about managing an analysis that changes.
Kai: Right. They are focusing on how to represent these evolving analyses as structured objects in a repository and then having a factory handle running those preserved states, which is where the actual computations happen.
Lev: From my side, I’m thinking this separation between the definition materials and the execution environment is key because when you move to real hardware, you don't want your analysis logic tied too tightly to a specific backend runner.
Mira: And they solve that by using this Yuki middleware layer to connect those preserved states to the execution infrastructure, which essentially builds those Snakemake workflows on the fly for whatever backend is available.
Kai: It sounds like they’re trying to make sure that even if we change how we run something tomorrow, the definition of *what* we analyzed stays consistent and traceable.
Lev: It really seems like they are tackling the problem of provenance head-on, making it possible to trace any final result back to a specific configuration of code and parameters.
The paper's summary: Kai: They dive into how this system works by detailing the workflow architecture, distinguishing between an algorithm—which is just *how* you perform a computation—and a task, which is that specific application of it with its inputs and dependencies.
Mira: And they emphasize that the workflow records dependencies among these materials needed to define the analysis, not just the order in which you run things temporally or logically. That’s important because development is iterative.
Lev: Because when you're iterating on a physics result, your code changes constantly, so having a structure where dependencies are defined by the materials themselves makes sense for tracking that evolution.
Kai: Then they introduce the impression mechanism, which is how they preserve immutable states of these objects along with the states of their dependencies. An impression is essentially a snapshot of an object at a particular analysis state, recording all its defining materials and references to impressions of its dependencies.
Mira: So, instead of just saving the final output file, you save an entire context—the code version, the parameters used, everything—so you can always reconstruct that exact analysis setup later.
Lev: That persistent identifier for each impression is crucial because it means we can unambiguously link any generated result back to that specific analysis state from which it originated.
Kai: It’s about making sure that when you look at a result, you know exactly which version of the analysis produced it, which is what they call provenance tracking.
The paper's improvements: Mira: The authors suggest a few key improvements to this concept. They focus on making the repository structure concrete by using four object types: project, directory, algorithm, and task to organize the materials needed for an analysis.
Kai: And they highlight that dependencies are expressed between these objects rather than just hard-coded paths to outputs. This lets tasks assign local aliases to upstream analysis components instead of relying on static file locations.
Lev: That sounds like a big step toward flexibility, because if you change the structure or move a directory around, the dependency definition itself is more resilient because it uses these internal object references.
Mira: They also emphasize that this separation allows the analysis representation to remain completely independent of any particular execution backend you might choose later on. This is what makes the system robust for long-term use across different computational platforms.
Kai: So, the core improvement here is achieving this balance: keeping the definition lightweight while retaining all the structural and computational configuration information needed for reproducibility.
Lev: It seems like they’ve really focused on making sure that whatever execution engine you plug in later, it just reads these preserved states and runs them correctly without needing to understand the internal structure of the analysis definition itself.
Conclusion: Kai: So, wrapping up, this paper on "CELEBI: Design and Realization of a Reproducible Physics Analysis Framework" provides a coherent way to represent an evolving physics analysis by separating the definition from execution and using those immutable impressions to track states.
Mira: The main implication is establishing a foundation for provenance tracking, which really helps with collaboration and long-term analysis preservation in high energy physics. It keeps the repository lightweight while retaining all the information about its logical structure and computational setup.
Lev: For us on the hardware side, it means we have a much clearer way to define what needs to be run without being locked into a specific execution backend, which is something we need as we build out more complex error correction systems.
Kai: It sets up a solid structure for reproducible execution across different teams and different hardware setups. It’s about making the analysis definition itself durable enough to survive changes in the analysis process.
Mira: Overall, it gives researchers a tool to maintain a coherent picture of an evolving physics analysis, which is vital for tracking results over many years. This framework paves the way for more reliable collaboration and long-term data preservation in this field.
Lev: And from my view, having that explicit connection between the preserved state and the execution infrastructure through Yuki middleware is what really makes it work for real hardware deployment. That bridge is important when you're dealing with complex error correction overheads.
Mingrui Zhao
School of Physics, State Key Laboratory of Nuclear Physics and Technology, Peking University
hep-ex
Submitted: 2026-11-01
Updated: 2026-11-01
Comments: 8 pages, 1 figures, CHEP proceedings
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 97/100
The gist: Reproducibility in modern physics analysis requires more than preserving source code or individual workflow descriptions<ref:2611.01788#pg3> Celebi…
Key concepts
- Analysis Repository
- This component stores the lightweight materials needed to define an analysis. It includes source code, parameters, configurations, and documentation. It keeps the analysis definition separate from the actual generated data products.
- Production Factory
- This is responsible for executing preserved analysis states and managing the resulting data products. It handles running computations based on stored definitions without needing to know the specific execution backend.
- Impression Mechanism
- An impression is an immutable representation of an object at a specific analysis state, including its dependencies. This mechanism preserves the exact configuration (code, parameters) and links generated results unambiguously back to the precise analysis state that produced them.
- Workflow Architecture
- This describes the analysis by defining materials required for computations and their dependencies. It distinguishes between an algorithm (how computation is done) and a task (a specific application of that algorithm with parameters). Dependencies are recorded among these materials, not just execution order.
Terminology
Summary
Reproducibility in modern physics analysis requires more than preserving source code or individual workflow descriptions<ref:2611. p. 0 Celebi: Design and Realization of a Reproducible Physics Analysis
Framework
Mingrui Zhao∗
School of Physics, State Key Laboratory of Nuclear Physics and Technology
Peking University, Beijing, China
Abstract
Reproducibility in modern physics analysis requires more than preserving source code or
individual workflow descriptions. An analysis evolves continuously through changes in code,
configuration, parameters, input datasets, and dependency structures, making it necessary to
maintain a coherent representation of both the analysis itself and the provenance of its results. An analysis evolves continuously through changes in code,
configuration, parameters, input datasets, and dependency structures, making it necessary to
maintain a coherent representation of both the analysis itself and the provenance of its results. We present Celebi, a framework for organizing and preserving evolving physics analyses
while keeping the analysis definition lightweight and separate from generated data products. This motivates an analysis-management layer that complements both workflow execution and
conventional version control. In this work, we introduce Celebi, an analysis-management framework designed to provide
such a representation. At the system level, Celebi maintains the analysis definition and its preserved states in an
Analysis Repository, while execution and generated results are handled separately through a
Production Factory. The middleware layer, Yuki, connects preserved analysis states to the
execution infrastructure by constructing executable Snakemake workflows and dispatching
them to supported workflow runners.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Framework Architecture
The Celebi system separates the representation and preservation of an analysis from the execution of the corresponding computations<ref:2611.01788#pg4> Its architecture consists of two main components: the Analysis Repository and the Production Factory, connected through the Yuki middleware<ref:2611. p. 2 2 Overview of the Celebi system
The Celebi system separates the representation and preservation of an analysis from the execution
of the corresponding computations. Its architecture consists of two main components: the
Analysis Repository and the Production Factory, connected through the Yuki middleware. Its architecture consists of two main components: the
Analysis Repository and the Production Factory, connected through the Yuki middleware. The Analysis Repository contains the lightweight materials required to define an analysis,
including source code, configurations, parameters, documentation, and dependency information. Selected states of these objects can be preserved as immutable impressions,
allowing particular analysis configurations to remain identifiable after the working analysis has
changed. The Production Factory is responsible for executing preserved analysis states and managing
the resulting data products. Yuki provides the interface between the Analysis Repository and the execution infrastructure. Given an impression submitted for execution, Yuki resolves its dependency structure and
constructs the corresponding Snakemake workflow. The middleware layer, Yuki, connects preserved analysis states to the
execution infrastructure by constructing executable Snakemake workflows and dispatching
them to supported workflow runners. This separation allows the analysis representation
to remain independent of a particular execution backend.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Analysis Representation Design
The design focuses on three closely related aspects: workflow architecture, file organization, and the impression mechanism<ref:2611.01788#pg5> The workflow architecture describes the analysis in terms of materials required to define each computation and their dependencies<ref:2611. p. 1 We introduce three
closely related aspects of this design. First, the workflow architecture describes the analysis
in terms of the materials required to define each computation and the dependencies among
them. A central
distinction in this model is made between an algorithm and a task. An algorithm describes how a
computation is performed, whereas a task describes a particular application of that computation
with specified parameters and upstream dependencies. Source code, computational procedures,
task-specific parameters, and dependency information are treated as explicit components of the
analysis representation. The workflow therefore describes relationships among the materials
that define the computation rather than only the temporal or logical ordering of execution
steps.Where the paper says it
Where the paper says it
Where the paper says it
State Preservation Mechanism
The impression mechanism preserves immutable states of objects together with their corresponding states of their dependencies<ref:2611.01788#pg5> An impression is an immutable representation of a Celebi object at a particular analysis state, recording materials defining the object along with references to impressions corresponding to its dependencies<ref:2611. p. 5 An impression is an immutable representation of a Celebi object at a particular analysis
state. It records the materials defining the object together with references to the impressions
corresponding to its dependencies. Each impression is assigned a persistent identifier that remains associated with that immutable
state. For an object with dependencies, the impression metadata records the identifiers of the
exact predecessor impressions on which it depends. A
structured file organization gives this model a concrete and human-readable realization in the
analysis repository, separating reusable algorithms, task-specific configurations, parameters,
documentation, and dependencies into well-defined objects. Finally, the impression mechanism
preserves immutable states of these objects together with the corresponding states of their
dependencies, providing persistent identities for particular analysis configurations and allowing
generated results to be associated unambiguously with the analysis state from which they
originate.Where the paper says it
Where the paper says it
Where the paper says it
Repository Structure Realization
The file organization realizes the workflow model in a Celebi repository using four object types: project, directory, algorithm, and task<ref:2611.01788#pg6> Project and directory objects provide the hierarchical organization of the repository<ref:2611. p. 4 They serve two
complementary roles. Project and directory objects provide the hierarchical organization of the
repository, while algorithm and task objects represent the materials that participate directly in
the workflow. They serve two
complementary roles. Project and directory objects provide the hierarchical organization of the
repository, while algorithm and task objects represent the materials that participate directly in
the workflow. Dependencies are expressed between objects rather than through hard-coded paths to
generated outputs. A task can assign local aliases to its dependencies, allowing its computational
definition to refer to upstream analysis components through stable logical names.Where the paper says it
Where the paper says it
Where the paper says it
Conclusion and Scope
The concepts introduced provide a coherent representation of an evolving physics analysis and establish a foundation for reproducibility, provenance tracking, collaboration, and long-term analysis preservation<ref:2611.01788#pg5> The framework allows the repository to remain lightweight while retaining the information required to describe its logical structure and computational configuration<ref:2611. p. 5 With this organization, the Analysis Repository contains the relatively small materials that
define the analysis rather than its potentially much larger generated products. The repository can
therefore remain lightweight as the analysis evolves, while still retaining the information required
to describe its logical structure and computational configuration. REANA is one currently supported execution platform. Together, the materialized workflow, structured repository organization, and impression
mechanism provide a foundation for provenance tracking, reproducible execution, collaborative
development, and long-term preservation of high-energy-physics analyses. Generated results and execution are handled separately through a Production Factory, with the
Yuki middleware providing the connection between preserved analysis states and the execution
infrastructure. The detailed implementation of this execution layer is outside the scope of this
paper. Execution is separated from this persistent analysis representation. The Yuki middleware
converts preserved analysis states into Snakemake workflows and dispatches them to supported
workflow runners, allowing different execution backends to be used without changing the
underlying analysis model.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
REFERENCES
[1] Monya Baker. 500 scientists lift the lid on reproducibility. p. 7 References
[1] Monya Baker. 1,500 scientists lift the lid on reproducibility.Where the paper says it
[2] Xiaoli Chen et al. Open is not enough. p. 7 [2] Xiaoli Chen et al. Open is not enough. [2] Xiaoli Chen et al. Open is not enough. Open is not enough. Nature Phys. Open is not enough. Nature Phys.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[3] Roman Kogler, David M. South, and Michael Steder. p. 7 [3] Roman Kogler, David M. South, and Michael Steder. The importance of analysis preservation and reproducibility has long been recognized within
the high-energy physics community. Community-wide efforts on data preservation were already
discussed systematically in Ref. The importance of analysis preservation and reproducibility has long been recognized within
the high-energy physics community. Community-wide efforts on data preservation were already
discussed systematically in Ref.Where the paper says it
Where the paper says it
Where the paper says it
[4] Xiaoli Chen et al. CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research. p. 7 [4] Xiaoli Chen et al. CERN Analysis Preservation: A Novel Digital Library Service to Enable
Reusable and Reproducible Research. [4] Xiaoli Chen et al. CERN Analysis Preservation: A Novel Digital Library Service to Enable
Reusable and Reproducible Research. CERN Analysis Preservation: A Novel Digital Library Service to Enable
Reusable and Reproducible Research. Notes Comput. CERN Analysis Preservation: A Novel Digital Library Service to Enable
Reusable and Reproducible Research. Notes Comput.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[5] Tibor Šimko et al. REANA: A System for Reusable Research Data Analyses. p. 7 [5] Tibor Šimko et al. REANA: A System for Reusable Research Data Analyses. [5] Tibor Šimko et al. REANA: A System for Reusable Research Data Analyses. REANA: A System for Reusable Research Data Analyses. EPJ Web
Conf. REANA: A System for Reusable Research Data Analyses. EPJ Web
Conf.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[6] Felix Mölder et al. Sustainable data analysis with Snakemake. p. 7 [6] Felix Mölder et al. Sustainable data analysis with Snakemake. [6] Felix Mölder et al. Sustainable data analysis with Snakemake.Where the paper says it
Where the paper says it
Barek 533:452–454, Nature, 2016
Chen et al. Open is not enough. Nature Phys., 15:113–119, 2019
Kogler et al. Data Preservation in High Energy Physics. J. Phys. Conf. Ser., 368:012026, 2012
Chen et al. CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research. Lect. Notes Comput. Sci., 9819:347–356, 2016
Šimko et al. REANA: A System for Reusable Research Data Analyses. EPJ Web Conf., 214:06034, 2019
Mölder et al. Sustainable data analysis with Snakemake.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Precise provenance tracking for generated results: The improved system can
identify the analysis state from which it originated, including the relevant code, configuration, parameters, inputs, and upstream dependencies
for any generated result using task impressions to bind results unambiguously to a specific analysis state. -
Decoupled execution environment management: The system can maintain
a coherent representation of both the analysis itself and the provenance of its results
while allowingexecution and generated results are handled separately through a Production Factory,
ensuring that the analysis definition remains independent of any particular execution backend. -
Explicit material-based workflow definition: The system can represent a materialized workflow where dependencies are recorded
among the materials required to define them,
distinguishing between an algorithm and a task to allow thesame computational procedure to be reused in different parts of an analysis, or with different inputs and parameter values.
-
Immutable state management for reproducibility: By using impressions, the system can preserve
immutable states of these objects together with the corresponding states of their dependencies,
providingpersistent identities for particular analysis configurations
even as the working analysis continues to change. -
Human-readable repository structure: The file organization allows for a concrete realization where
Project and directory objects provide the hierarchical organization of the repository,
offering adirect and human-readable representation
of the logical analysis structure within the Analysis Repository.
Abstract
Reproducibility in modern physics analysis requires more than preserving source code or workflow descriptions. Analyses evolve through changes in code, configurations, parameters, input datasets, and dependencies, requiring a coherent representation of both the analysis and the provenance of its results. We present Celebi, a framework for organizing and preserving evolving physics analyses while keeping analysis definitions lightweight and separate from generated data products. Celebi maintains analysis definitions and preserved states in an Analysis Repository, while execution and generated results are managed separately through a Production Factory. The middleware layer, Yuki, connects preserved analysis states to execution infrastructure by constructing executable Snakemake workflows and dispatching them to supported runners, keeping the analysis representation independent of execution backends. This paper presents three main design concepts. First, the workflow architecture provides a modular representation of analysis components and their dependencies. Second, a structured file organization realizes this model in a human-readable form, separating reusable algorithms, task-specific configurations, parameters, documentation, and dependencies into well-defined objects. Third, the impression mechanism preserves immutable states of these objects and their dependencies, assigning persistent identities to specific analysis configurations and linking generated results unambiguously to their originating states. Together, these concepts establish a coherent representation of evolving physics analyses, supporting reproducibility, provenance tracking, collaboration, and long-term preservation.
Related papers
- DrSai: an agentic AI system toward autonomous physics analysis workflows at BESIII
- CANTON- mu Proposal: A Next-Generation Muon g-2 Measurement at Sub-0.1 ppm Precision
- Search for dark photon dark matter near 1,meV using a superconducting mixer
- ANUBIS detector and its sensitivity to neutral long-lived particles
- How to scale your HEP ML models: A recipe for robust architecture comparisons at scale
- Search for the 16 O(ppp) to 13 C pi+ pi+ e+ Decay Mode in Super-Kamiokande Using Machine Learning Techniques