CELEBI: Design and Realization of a Reproducible Physics Analysis Framework

arXiv:2611.01788 · hep-ex · Submitted 2026-11-01 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: High Energy Physics: Experiment. The day in high-energy experiment: detectors, searches and limits.

Kai: We're looking at the paper: "CELEBI: Design and Realization of a Reproducible Physics Analysis Framework".

Mira: Here's the gist of the paper. Reproducibility in modern physics analysis requires more than preserving source code or individual workflow descriptions<ref:2611.01788#pg3> Celebi, a framework for organizing and preserving evolving physics analyses while keeping the analysis definition lightweight and separate from generated data products<ref:2611.01788#pg6> The paper introduces Celebi, an analysis-management framework designed to provide such a representation<ref:2611.01788#pg4> It achieves this by separating the analysis definition from generated data products into an Analysis Repository and a Production Factory connected by Yuki middleware<ref:2611.01788#pg3> Framework Architecture The Celebi system separates the representation and preservation of an analysis from the execution of the corresponding computations<ref:2611.01788#pg4> Its architecture consists of two main components: the Analysis Repository and the Production Factory, connected through the Yuki middleware<ref:2611.01788#pg4> The Analysis Repository contains lightweight materials required to define an analysis, including source code, configurations, parameters, documentation, and dependency information<ref:2611.01788#pg5> The Production Factory is responsible for executing preserved analysis states and managing the resulting data products<ref:2611.01788#pg4> Yuki provides the interface between the Analysis Repository and the execution infrastructure, resolving dependency structures to construct Snakemake workflows<ref:2611.01788#pg4> This separation allows the analysis representation to remain independent of a particular execution backend<ref:2611.01788#pg4> Analysis Representation Design The design focuses on three closely related aspects: workflow architecture, file organization, and the impression mechanism<ref:2611.01788#pg5> The workflow architecture describes the analysis in terms of materials required to define each computation and their dependencies<ref:2611.01788#pg5> It distinguishes between an algorithm, which describes how a computation is performed, and a task, which describes a particular application of that computation with specified parameters and upstream dependencies<ref:2611.01788#pg5> The workflow records dependencies among the materials required to define the analysis rather than only the temporal or logical ordering of execution steps<ref:2611.01788#pg5> State Preservation Mechanism The impression mechanism preserves immutable states of objects together with their corresponding states of their dependencies<ref:2611.01788#pg5> An impression is an immutable representation of a Celebi object at a particular analysis state, recording materials defining the object along with references to impressions corresponding to its dependencies<ref:2611.01788#pg7> Each impression is assigned a persistent identifier that remains associated with that immutable state<ref:2611.01788#pg8> This mechanism allows generated results to be associated unambiguously with the analysis state from which they originated<ref:2611.01788#pg5> Repository Structure Realization The file organization realizes the workflow model in a Celebi repository using four object types: project, directory, algorithm, and task<ref:2611.01788#pg6> Project and directory objects provide the hierarchical organization of the repository<ref:2611.01788#pg6> Algorithm and task objects represent the materials that participate directly in the workflow<ref:2611.01788#pg6> Dependencies are expressed between objects rather than through hard-coded paths to generated outputs, allowing tasks to assign local aliases to upstream analysis components<ref:2611.01788#pg6> The concepts introduced provide a coherent representation of an evolving physics analysis and establish a foundation for reproducibility, provenance tracking, collaboration, and long-term analysis preservation<ref:2611.01788#pg5> The framework allows the repository to remain lightweight while retaining the information required to describe its logical structure and computational configuration<ref:2611.01788#pg6> Together, these elements provide a foundation for provenance tracking, reproducible execution, collaborative development, and long-term preservation of high-energy-physics analyses<ref:2611.01788#pg8> The detailed implementation of the execution layer is outside the scope of this paper<ref:2611.01788#pg4> The final architecture supports different execution backends without changing the underlying analysis model<ref:2611.01788#pg4> REFERENCES

1: Monya Baker.

Kai: 500 scientists lift the lid on reproducibility.

Mira: Nature, 533:452–454, 2016

2: Xiaoli Chen et al.

Kai: Open is not enough.

Title and authors: Mira: So, to frame what we’re hearing today, "CELEBI: Design and Realization of a Reproducible Physics Analysis Framework" is about building this whole system where the definition stays separate from the execution. It's not just a better way to save files; it's about managing an analysis that changes.

Kai: Right. They are focusing on how to represent these evolving analyses as structured objects in a repository and then having a factory handle running those preserved states, which is where the actual computations happen.

Lev: From my side, I’m thinking this separation between the definition materials and the execution environment is key because when you move to real hardware, you don't want your analysis logic tied too tightly to a specific backend runner.

Mira: And they solve that by using this Yuki middleware layer to connect those preserved states to the execution infrastructure, which essentially builds those Snakemake workflows on the fly for whatever backend is available.

Kai: It sounds like they’re trying to make sure that even if we change how we run something tomorrow, the definition of *what* we analyzed stays consistent and traceable.

Lev: It really seems like they are tackling the problem of provenance head-on, making it possible to trace any final result back to a specific configuration of code and parameters.

The paper's summary: Kai: They dive into how this system works by detailing the workflow architecture, distinguishing between an algorithm—which is just *how* you perform a computation—and a task, which is that specific application of it with its inputs and dependencies.

Mira: And they emphasize that the workflow records dependencies among these materials needed to define the analysis, not just the order in which you run things temporally or logically. That’s important because development is iterative.

Lev: Because when you're iterating on a physics result, your code changes constantly, so having a structure where dependencies are defined by the materials themselves makes sense for tracking that evolution.

Kai: Then they introduce the impression mechanism, which is how they preserve immutable states of these objects along with the states of their dependencies. An impression is essentially a snapshot of an object at a particular analysis state, recording all its defining materials and references to impressions of its dependencies.

Mira: So, instead of just saving the final output file, you save an entire context—the code version, the parameters used, everything—so you can always reconstruct that exact analysis setup later.

Lev: That persistent identifier for each impression is crucial because it means we can unambiguously link any generated result back to that specific analysis state from which it originated.

Kai: It’s about making sure that when you look at a result, you know exactly which version of the analysis produced it, which is what they call provenance tracking.

The paper's improvements: Mira: The authors suggest a few key improvements to this concept. They focus on making the repository structure concrete by using four object types: project, directory, algorithm, and task to organize the materials needed for an analysis.

Kai: And they highlight that dependencies are expressed between these objects rather than just hard-coded paths to outputs. This lets tasks assign local aliases to upstream analysis components instead of relying on static file locations.

Lev: That sounds like a big step toward flexibility, because if you change the structure or move a directory around, the dependency definition itself is more resilient because it uses these internal object references.

Mira: They also emphasize that this separation allows the analysis representation to remain completely independent of any particular execution backend you might choose later on. This is what makes the system robust for long-term use across different computational platforms.

Kai: So, the core improvement here is achieving this balance: keeping the definition lightweight while retaining all the structural and computational configuration information needed for reproducibility.

Lev: It seems like they’ve really focused on making sure that whatever execution engine you plug in later, it just reads these preserved states and runs them correctly without needing to understand the internal structure of the analysis definition itself.

Conclusion: Kai: So, wrapping up, this paper on "CELEBI: Design and Realization of a Reproducible Physics Analysis Framework" provides a coherent way to represent an evolving physics analysis by separating the definition from execution and using those immutable impressions to track states.

Mira: The main implication is establishing a foundation for provenance tracking, which really helps with collaboration and long-term analysis preservation in high energy physics. It keeps the repository lightweight while retaining all the information about its logical structure and computational setup.

Lev: For us on the hardware side, it means we have a much clearer way to define what needs to be run without being locked into a specific execution backend, which is something we need as we build out more complex error correction systems.

Kai: It sets up a solid structure for reproducible execution across different teams and different hardware setups. It’s about making the analysis definition itself durable enough to survive changes in the analysis process.

Mira: Overall, it gives researchers a tool to maintain a coherent picture of an evolving physics analysis, which is vital for tracking results over many years. This framework paves the way for more reliable collaboration and long-term data preservation in this field.

Lev: And from my view, having that explicit connection between the preserved state and the execution infrastructure through Yuki middleware is what really makes it work for real hardware deployment. That bridge is important when you're dealing with complex error correction overheads.

Mingrui Zhao

School of Physics, State Key Laboratory of Nuclear Physics and Technology, Peking University

hep-ex

Submitted: 2026-11-01

Updated: 2026-11-01

Comments: 8 pages, 1 figures, CHEP proceedings

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 97/100

The gist: Reproducibility in modern physics analysis requires more than preserving source code or individual workflow descriptions<ref:2611.01788#pg3> Celebi…

Key concepts

Analysis Repository
This component stores the lightweight materials needed to define an analysis. It includes source code, parameters, configurations, and documentation. It keeps the analysis definition separate from the actual generated data products.
Production Factory
This is responsible for executing preserved analysis states and managing the resulting data products. It handles running computations based on stored definitions without needing to know the specific execution backend.
Impression Mechanism
An impression is an immutable representation of an object at a specific analysis state, including its dependencies. This mechanism preserves the exact configuration (code, parameters) and links generated results unambiguously back to the precise analysis state that produced them.
Workflow Architecture
This describes the analysis by defining materials required for computations and their dependencies. It distinguishes between an algorithm (how computation is done) and a task (a specific application of that algorithm with parameters). Dependencies are recorded among these materials, not just execution order.

Terminology

Summary

Reproducibility in modern physics analysis requires more than preserving source code or individual workflow descriptions<ref:2611. p. 0

Where the paper says it

Celebi: Design and Realization of a Reproducible Physics Analysis Framework Mingrui Zhao∗ School of Physics, State Key Laboratory of Nuclear Physics and Technology Peking University, Beijing, China Abstract Reproducibility in modern physics analysis requires more than preserving source code or individual workflow descriptions. An analysis evolves continuously through changes in code, configuration, parameters, input datasets, and dependency structures, making it necessary to maintain a coherent representation of both the analysis itself and the provenance of its results.

01788#pg3> Celebi, a framework for organizing and preserving evolving physics analyses while keeping the analysis definition lightweight and separate from generated data products<ref:2611. p. 0
Where the paper says it

An analysis evolves continuously through changes in code, configuration, parameters, input datasets, and dependency structures, making it necessary to maintain a coherent representation of both the analysis itself and the provenance of its results. We present Celebi, a framework for organizing and preserving evolving physics analyses while keeping the analysis definition lightweight and separate from generated data products.

01788#pg6> The paper introduces Celebi, an analysis-management framework designed to provide such a representation<ref:2611. p. 1
Where the paper says it

This motivates an analysis-management layer that complements both workflow execution and conventional version control. In this work, we introduce Celebi, an analysis-management framework designed to provide such a representation.

01788#pg4> It achieves this by separating the analysis definition from generated data products into an Analysis Repository and a Production Factory connected by Yuki middleware<ref:2611. p. 0
Where the paper says it

At the system level, Celebi maintains the analysis definition and its preserved states in an Analysis Repository, while execution and generated results are handled separately through a Production Factory. The middleware layer, Yuki, connects preserved analysis states to the execution infrastructure by constructing executable Snakemake workflows and dispatching them to supported workflow runners.

01788#pg3>

Framework Architecture

The Celebi system separates the representation and preservation of an analysis from the execution of the corresponding computations<ref:2611.01788#pg4> Its architecture consists of two main components: the Analysis Repository and the Production Factory, connected through the Yuki middleware<ref:2611. p. 2

Where the paper says it

2 Overview of the Celebi system The Celebi system separates the representation and preservation of an analysis from the execution of the corresponding computations. Its architecture consists of two main components: the Analysis Repository and the Production Factory, connected through the Yuki middleware.

01788#pg4> The Analysis Repository contains lightweight materials required to define an analysis, including source code, configurations, parameters, documentation, and dependency information<ref:2611. p. 2
Where the paper says it

Its architecture consists of two main components: the Analysis Repository and the Production Factory, connected through the Yuki middleware. The Analysis Repository contains the lightweight materials required to define an analysis, including source code, configurations, parameters, documentation, and dependency information.

01788#pg5> The Production Factory is responsible for executing preserved analysis states and managing the resulting data products<ref:2611. p. 2
Where the paper says it

Selected states of these objects can be preserved as immutable impressions, allowing particular analysis configurations to remain identifiable after the working analysis has changed. The Production Factory is responsible for executing preserved analysis states and managing the resulting data products.

01788#pg4> Yuki provides the interface between the Analysis Repository and the execution infrastructure, resolving dependency structures to construct Snakemake workflows<ref:2611. p. 2
Where the paper says it

Yuki provides the interface between the Analysis Repository and the execution infrastructure. Given an impression submitted for execution, Yuki resolves its dependency structure and constructs the corresponding Snakemake workflow.

01788#pg4> This separation allows the analysis representation to remain independent of a particular execution backend<ref:2611. p. 0
Where the paper says it

The middleware layer, Yuki, connects preserved analysis states to the execution infrastructure by constructing executable Snakemake workflows and dispatching them to supported workflow runners. This separation allows the analysis representation to remain independent of a particular execution backend.

01788#pg4>

Analysis Representation Design

The design focuses on three closely related aspects: workflow architecture, file organization, and the impression mechanism<ref:2611.01788#pg5> The workflow architecture describes the analysis in terms of materials required to define each computation and their dependencies<ref:2611. p. 1

Where the paper says it

We introduce three closely related aspects of this design. First, the workflow architecture describes the analysis in terms of the materials required to define each computation and the dependencies among them.

01788#pg5> It distinguishes between an algorithm, which describes how a computation is performed, and a task, which describes a particular application of that computation with specified parameters and upstream dependencies<ref:2611. p. 3
Where the paper says it

A central distinction in this model is made between an algorithm and a task. An algorithm describes how a computation is performed, whereas a task describes a particular application of that computation with specified parameters and upstream dependencies.

01788#pg5> The workflow records dependencies among the materials required to define the analysis rather than only the temporal or logical ordering of execution steps<ref:2611. p. 3
Where the paper says it

Source code, computational procedures, task-specific parameters, and dependency information are treated as explicit components of the analysis representation. The workflow therefore describes relationships among the materials that define the computation rather than only the temporal or logical ordering of execution steps.

01788#pg5>

State Preservation Mechanism

The impression mechanism preserves immutable states of objects together with their corresponding states of their dependencies<ref:2611.01788#pg5> An impression is an immutable representation of a Celebi object at a particular analysis state, recording materials defining the object along with references to impressions corresponding to its dependencies<ref:2611. p. 5

Where the paper says it

An impression is an immutable representation of a Celebi object at a particular analysis state. It records the materials defining the object together with references to the impressions corresponding to its dependencies.

01788#pg7> Each impression is assigned a persistent identifier that remains associated with that immutable state<ref:2611. p. 6
Where the paper says it

Each impression is assigned a persistent identifier that remains associated with that immutable state. For an object with dependencies, the impression metadata records the identifiers of the exact predecessor impressions on which it depends.

01788#pg8> This mechanism allows generated results to be associated unambiguously with the analysis state from which they originated<ref:2611. p. 0
Where the paper says it

A structured file organization gives this model a concrete and human-readable realization in the analysis repository, separating reusable algorithms, task-specific configurations, parameters, documentation, and dependencies into well-defined objects. Finally, the impression mechanism preserves immutable states of these objects together with the corresponding states of their dependencies, providing persistent identities for particular analysis configurations and allowing generated results to be associated unambiguously with the analysis state from which they originate.

01788#pg5>

Repository Structure Realization

The file organization realizes the workflow model in a Celebi repository using four object types: project, directory, algorithm, and task<ref:2611.01788#pg6> Project and directory objects provide the hierarchical organization of the repository<ref:2611. p. 4

Where the paper says it

They serve two complementary roles. Project and directory objects provide the hierarchical organization of the repository, while algorithm and task objects represent the materials that participate directly in the workflow.

01788#pg6> Algorithm and task objects represent the materials that participate directly in the workflow<ref:2611. p. 4
Where the paper says it

They serve two complementary roles. Project and directory objects provide the hierarchical organization of the repository, while algorithm and task objects represent the materials that participate directly in the workflow.

01788#pg6> Dependencies are expressed between objects rather than through hard-coded paths to generated outputs, allowing tasks to assign local aliases to upstream analysis components<ref:2611. p. 5
Where the paper says it

Dependencies are expressed between objects rather than through hard-coded paths to generated outputs. A task can assign local aliases to its dependencies, allowing its computational definition to refer to upstream analysis components through stable logical names.

01788#pg6>

Conclusion and Scope

The concepts introduced provide a coherent representation of an evolving physics analysis and establish a foundation for reproducibility, provenance tracking, collaboration, and long-term analysis preservation<ref:2611.01788#pg5> The framework allows the repository to remain lightweight while retaining the information required to describe its logical structure and computational configuration<ref:2611. p. 5

Where the paper says it

With this organization, the Analysis Repository contains the relatively small materials that define the analysis rather than its potentially much larger generated products. The repository can therefore remain lightweight as the analysis evolves, while still retaining the information required to describe its logical structure and computational configuration.

01788#pg6> Together, these elements provide a foundation for provenance tracking, reproducible execution, collaborative development, and long-term preservation of high-energy-physics analyses<ref:2611. p. 7
Where the paper says it

REANA is one currently supported execution platform. Together, the materialized workflow, structured repository organization, and impression mechanism provide a foundation for provenance tracking, reproducible execution, collaborative development, and long-term preservation of high-energy-physics analyses.

01788#pg8> The detailed implementation of the execution layer is outside the scope of this paper<ref:2611. p. 1
Where the paper says it

Generated results and execution are handled separately through a Production Factory, with the Yuki middleware providing the connection between preserved analysis states and the execution infrastructure. The detailed implementation of this execution layer is outside the scope of this paper.

01788#pg4> The final architecture supports different execution backends without changing the underlying analysis model<ref:2611. p. 7
Where the paper says it

Execution is separated from this persistent analysis representation. The Yuki middleware converts preserved analysis states into Snakemake workflows and dispatches them to supported workflow runners, allowing different execution backends to be used without changing the underlying analysis model.

01788#pg4>

REFERENCES

[1] Monya Baker. 500 scientists lift the lid on reproducibility. p. 7

Where the paper says it

References [1] Monya Baker. 1,500 scientists lift the lid on reproducibility.

Nature, 533:452–454, 2016

[2] Xiaoli Chen et al. Open is not enough. p. 7

Where the paper says it

[2] Xiaoli Chen et al. Open is not enough.

p. 7
Where the paper says it

[2] Xiaoli Chen et al. Open is not enough.

Nature Phys. p. 7
Where the paper says it

Open is not enough. Nature Phys.

p. 7
Where the paper says it

Open is not enough. Nature Phys.

, 15:113–119, 2019

[3] Roman Kogler, David M. South, and Michael Steder. p. 7

Where the paper says it

[3] Roman Kogler, David M. South, and Michael Steder.

Data Preservation in High Energy Physics. p. 1
Where the paper says it

The importance of analysis preservation and reproducibility has long been recognized within the high-energy physics community. Community-wide efforts on data preservation were already discussed systematically in Ref.

p. 1
Where the paper says it

The importance of analysis preservation and reproducibility has long been recognized within the high-energy physics community. Community-wide efforts on data preservation were already discussed systematically in Ref.

J. Phys. Conf. Ser., 368:012026, 2012

[4] Xiaoli Chen et al. CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research. p. 7

Where the paper says it

[4] Xiaoli Chen et al. CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research.

p. 7
Where the paper says it

[4] Xiaoli Chen et al. CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research.

Lect. Notes Comput. p. 7
Where the paper says it

CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research. Notes Comput.

p. 7
Where the paper says it

CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research. Notes Comput.

Sci., 9819:347–356, 2016

[5] Tibor Šimko et al. REANA: A System for Reusable Research Data Analyses. p. 7

Where the paper says it

[5] Tibor Šimko et al. REANA: A System for Reusable Research Data Analyses.

p. 7
Where the paper says it

[5] Tibor Šimko et al. REANA: A System for Reusable Research Data Analyses.

EPJ Web Conf. p. 7
Where the paper says it

REANA: A System for Reusable Research Data Analyses. EPJ Web Conf.

p. 7
Where the paper says it

REANA: A System for Reusable Research Data Analyses. EPJ Web Conf.

, 214:06034, 2019

[6] Felix Mölder et al. Sustainable data analysis with Snakemake. p. 7

Where the paper says it

[6] Felix Mölder et al. Sustainable data analysis with Snakemake.

p. 7
Where the paper says it

[6] Felix Mölder et al. Sustainable data analysis with Snakemake.

F1000Research, 10:33, 2025

Barek 533:452–454, Nature, 2016

Chen et al. Open is not enough. Nature Phys., 15:113–119, 2019

Kogler et al. Data Preservation in High Energy Physics. J. Phys. Conf. Ser., 368:012026, 2012

Chen et al. CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research. Lect. Notes Comput. Sci., 9819:347–356, 2016

Šimko et al. REANA: A System for Reusable Research Data Analyses. EPJ Web Conf., 214:06034, 2019

Mölder et al. Sustainable data analysis with Snakemake.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Precise provenance tracking for generated results: The improved system can identify the analysis state from which it originated, including the relevant code, configuration, parameters, inputs, and upstream dependencies for any generated result using task impressions to bind results unambiguously to a specific analysis state.

  2. Decoupled execution environment management: The system can maintain a coherent representation of both the analysis itself and the provenance of its results while allowing execution and generated results are handled separately through a Production Factory, ensuring that the analysis definition remains independent of any particular execution backend.

  3. Explicit material-based workflow definition: The system can represent a materialized workflow where dependencies are recorded among the materials required to define them, distinguishing between an algorithm and a task to allow the same computational procedure to be reused in different parts of an analysis, or with different inputs and parameter values.

  4. Immutable state management for reproducibility: By using impressions, the system can preserve immutable states of these objects together with the corresponding states of their dependencies, providing persistent identities for particular analysis configurations even as the working analysis continues to change.

  5. Human-readable repository structure: The file organization allows for a concrete realization where Project and directory objects provide the hierarchical organization of the repository, offering a direct and human-readable representation of the logical analysis structure within the Analysis Repository.

Abstract

Reproducibility in modern physics analysis requires more than preserving source code or workflow descriptions. Analyses evolve through changes in code, configurations, parameters, input datasets, and dependencies, requiring a coherent representation of both the analysis and the provenance of its results. We present Celebi, a framework for organizing and preserving evolving physics analyses while keeping analysis definitions lightweight and separate from generated data products. Celebi maintains analysis definitions and preserved states in an Analysis Repository, while execution and generated results are managed separately through a Production Factory. The middleware layer, Yuki, connects preserved analysis states to execution infrastructure by constructing executable Snakemake workflows and dispatching them to supported runners, keeping the analysis representation independent of execution backends. This paper presents three main design concepts. First, the workflow architecture provides a modular representation of analysis components and their dependencies. Second, a structured file organization realizes this model in a human-readable form, separating reusable algorithms, task-specific configurations, parameters, documentation, and dependencies into well-defined objects. Third, the impression mechanism preserves immutable states of these objects and their dependencies, assigning persistent identities to specific analysis configurations and linking generated results unambiguously to their originating states. Together, these concepts establish a coherent representation of evolving physics analyses, supporting reproducibility, provenance tracking, collaboration, and long-term preservation.

Related papers