Open Science with the LHCb Ntupling Service
summary
The gist
The LHCb Ntupling Service presents an innovative methodology for providing open access to data collected in HEP experiments by allowing on-demand production of custom LHCb open data.
In short
The LHCb Ntupling Service is a web service enabling users to create and produce custom LHCb open data on demand. It allows for flexible exploration of physics selections without requiring experiment-specific software knowledge. The system streamlines data request tracking and provides a direct communication channel with the LHCb Open Data team, significantly lowering the barrier for new researchers to access vast datasets from Run 1 and Run 2 collisions.
Key concepts
- LHCb Ntupling Service
- This is a web service designed to let users create custom LHCb data on demand. Instead of needing complex software, users use a wizard to specify what kind of data they want, which the service then produces for them. It makes accessing experimental results much easier.
- LHCb Ntuple Wizard
- This is the user interface that guides researchers in selecting specific datasets and physics objects they are interested in. It helps users create configuration files (YAML) needed to generate their desired data, removing the need for users to learn complex production software like DaVinci.
- GitLab Integration
- The service uses GitLab to manage all requests. Each user request becomes an issue in a private repository, and its progress is tracked using labels like 'Awaiting LHCb review'. This provides a transparent workflow for managing data requests and audits.
- FAIR Principles Promotion
- This process allows users to formally promote their generated ntuples to permanent records on the CERN Open Data Portal. Once approved, these records receive a DOI, ensuring the data is discoverable and reusable according to FAIR principles.
Terminology used across episodes
This episode discusses
- Open Science with the LHCb Ntupling Service · Paper Radio
- Mini-AOD: A New Analysis Data Format for CMS
- Software and computing for Run 3 of the ATLAS experiment at the LHC
- Ntuple Wizard: An Application to Access Large-Scale Open Data from LHCb
- The LHCb Sprucing and Analysis Productions
- LHCb Open Data Ntupling Service: On-demand production and publishing of custom LHCb Open Data
- ROOT - A C++ Framework for Petabyte Data Storage, Statistical Analysis and Visualization
- Using CMS Open Data in research -- challenges and directions
- The LHCb Stripping Project: Sustainable Legacy Data Processing for High-Energy Physics
- Status Report of the DPHEP Study Group: Towards a Global Effort for Sustainable Data Preservation in High Energy Physics
- FunTuple: A new N-tuple component for offline data processing at the LHCb experiment
The paper
Open Science with the LHCb Ntupling Service · Read on arXiv
Christine A. Aidala, Michael Buchar, Chris Burr, Dillon S. Fitzgerald, Cameron Duncan McClymont, Adam Morris, Sebastian Neubert, Piet Nogga, Ryunosuke O’Neil, Eduardo Rodrigues, Daan Eduard Rosendal, Tibor Simko
University of Michigan, United States · CERN, Switzerland · Vilnius University, Lithuania · University of Bonn, Germany · Oliver Lodge Laboratory, University of Liverpool, United Kingdom · Lakera company
The data samples collected at the Large Hadron Collider at CERN are highly valuable, and the analysis of these data has led to numerous discoveries and the 2025 Breakthrough Prize in Fundamental Physics. Preserving and providing open access to these data for future analysts is imperative to maximize the scientific output of the collected data. The LHCb collaboration has developed a novel approach to this pursuit of open science, the LHCb Ntupling Service: a web service for on-demand production and publishing of custom LHCb open data, providing open access to both Run 1, and for the first time, Run 2 proton-proton collision data collected by the LHCb experiment, amounting to over 4 PB of data to explore.
Transcript
Introduction to the show: ident: High Energy Physics: Experiment. The day in high-energy experiment: detectors, searches and limits.
Kai: We're looking at the paper: "Open Science with the LHCb Ntupling Service".
Mira: Here's the gist of the paper. The LHCb Ntupling Service presents an innovative methodology for providing open access to data collected in HEP experiments by allowing on-demand production of custom LHCb open data.
Kai: This service is significant because it offers flexibility in exploring physics selections while reducing the barrier to entry for new users without the need for experiment-specific software.
Mira: The Problem of Data Access Contemporary research in high energy physics (HEP) necessitates vast data volumes stored in custom file formats to meet the needs of experimental programs.
Kai: Both the complexity and sheer volumes of these datasets make it challenging to preserve and provide open access to these data.
Title and authors: Kai: The paper title is "Open Science with the LHCb Ntupling Service", and it’s authored by Christine Aidala, Michael Buchar, Chris Burr, Dillon S. Fitzgerald, Cameron Duncan McClymont, Adam Morris, Sebastian Neubert, Piet Nogga, Ryunosuke O’Neil Eduardo Rodrigues Daan Eduard Rosendal Tibor Simko.
Mira: It really just frames the whole thing as a way to make the data more open and accessible for everyone interested in high energy physics. They are pushing for FAIR principles for this data output >
Lev: What I find interesting about the authors is that they’re coming from different areas—like experimentalists, theorists, and error correction—which suggests a broad view of what’s needed to make this work.
Kai: Right, it shows how different parts of the physics community need to collaborate on these kinds of infrastructure projects.
Mira: It sets up the idea that we should be aiming for consistent standards across all the LHC experiments, not just one silo >
The paper's summary: Kai: They summarize this as a novel approach: a web service for on-demand production and publishing of custom LHCb open data. It addresses the challenge of preserving and providing open access to vast data volumes in custom file formats.
Mira: So, the core idea is that instead of users having to deal with all those complicated parsing tools, they can use this service to generate exactly what they need >
Lev: The paper mentions that it provides a single entry point for users to submit ntuple production jobs, track their requests, communicate with the LHCb Open Data team, and download their resulting output files.
Kai: That tracking part is important; they model the lifecycle using GitLab issue labels like "Awaiting LHCb review" and "Running test production".
Mira: And they talk about the data storage: they use a CephFS file share for user profiles and comments, and EOS directories for those large ntuple output files >
Lev: The big thing mentioned is that requests can be promoted to permanent records on the CERN Open Data Portal where a DOI is attributed to the record, which helps with FAIR principles >
The paper's improvements: Kai: One improvement they highlight is the use of an AI system combined with the LHCb Ntuple Wizard. This wizard guides users through selecting datasets and physics objects of interest, then generates YAML files for a configuration client-side >
Mira: That’s key because it reduces the barrier to entry; it removes the need for a user to learn experiment-specific software just to start producing ntuples >
Lev: And security is a big point they make: since the configurations are captured in static YAML files, the backend itself generates the necessary Python production files from those YAML files >
Kai: That means no malicious code can be injected into the WLCG or the LHCb file system because what runs in Analysis Productions originates from that service >
Mira: They also mention a way to get even more interaction: providing a request-specific communication channel between the user and the LHCb Open Data team so users can raise questions directly about their specific request >
Conclusion: Kai: So, to wrap up, this paper describes how the LHCb Ntupling Service provides access to Run one and for the first time Run two pp data through an on-demand system <ref:2610.28296#pg1,Run 1 and for the first time>. It has this whole workflow from wizard selection to output retrieval >
Mira: The big implication is that it lets researchers explore unique data from LHCb without needing to learn all the experiment-specific software first, which lowers that barrier significantly >
Lev: From a real hardware standpoint, the fact that this relies on a backend generating files from static YAML suggests the overhead lands in managing those requests rather than in running complex analysis jobs directly on our systems >
Kai: It’s a significant step for open science because it makes exploring this massive dataset much more accessible to new users who might otherwise be intimidated by the file formats >
Mira: This work, "Open Science with the LHCb Ntupling Service", is a big milestone for both LHCb and general open science because it lets us explore that data without needing deep, experiment-specific knowledge first.
Lev: I think the service’s design shows a clear path forward for making these massive datasets usable in a more standardized, on-demand manner >
More episodes
- Daily Summary for 2026-11-19
- 2512.21290-Model-independent ZH production cross section at FCC-ee
- 2610.21982-First Observation of the Radiative Decay D 0 to gamma omega
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- Daily Summary for 2026-11-18
- 2512.08065-Searches for Light Dark Matter and Evidence of Coherent Elastic Neutrino-Nucleus Scattering of Solar Neutrinos with the LUX-ZEPLIN (LZ) Experiment
- Daily Summary for 2026-11-18
- 2508.18022-Precision Measurements of the Electroweak Mixing Angle in the Region of the Z pole at the Large Hadron Collider
- 2512.23992-Prospect for measurement of CP-violating parameters of B s 0 to phi gamma at the Tera Z factory