Readout chain of SPD DAQ system

arXiv:2610.16515 · physics.ins-det · Submitted 2026-10-12 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Readout chain of SPD DAQ system".

Mira: Here's the gist of the paper. The Spin Physics Detector (SPD) experiment at the NICA collider aims to study the spin structure of the nucleon and spin-related phenomena in interactions of polarized proton and deuteron beams The experimental conditions, including the high interaction rate and the large number of readout channels, require a triggerless data acquisition system with high throughput and a scalable architecture DAQ System Architecture The readout chain of the SPD DAQ system is organized into two hierarchical levels for receiving, aggregating, and transmitting data from the FEE to the computing system As shown in figure 1, data from the FEE are transmitted to L1 concentrators, where several input streams are combined into one The aggregated data are then transmitted to L2 concentrators, which combine streams from several L1 concentrators and forward them to the computing system through a PCIe interface using direct memory access (DMA) The general DAQ architecture for the SPD experiment was proposed in

2: , including a hierarchical structure of concentrators, data-flow organization principles, and a White Rabbit-based time-synchronization system L1 Concentrator Functionality The first-level concentrator (L1) is an intermediate node of the readout chain that connects the FEE to the second-level concentrator (L2) The L1 concentrator is based on an Intel Cyclone 10 GX FPGA, which implements data processing and transmission, FEE interface control, interaction with the synchronization system, and network communication with L2 Communication between the L1 concentrator and the FEE is implemented using a copper differential interface The interface supports bidirectional communication between L1 and the FEE L2 Aggregation Strategy The second-level concentrator (L2) combines data streams from several first-level concentrators (L1) and transmits the aggregated data to the Readout PC L2 performs the aggregation of multiple input streams in hardware and forms the data stream for the computing system This organization shifts data reception, buffering, and aggregation from the software network subsystem to the FPGA hardware L2 uses Scatter-Gather DMA, which allows data located in several separate DDR4 memory regions to be transferred as part of a sequence of DMA operations Performance and Scalability Metrics The expected number of FEE-L1 links for different detector configurations is given in table 1 The minimum number of concentrators at each level is determined by the total number of FEE outputs Nout, where NL1 = Nout/8 and NL2 = NL1/16 The physical line rate of each optical channel is 10.3125 Gb/s, corresponding to a total physical line rate of 165 Gb/s for 16 channels The effective payload data rate for 16 channels varies from 87.6 Gb/s (11.0 GB/s) at the minimum payload size to 156.0 Gb/s (19.5 GB/s) at the maximum Ethernet payload size Control and Monitoring Path A separate Control Logic block handles control commands, configuration data, and monitoring information exchanged between the external control network and the L2, L1, and FEE nodes The highspeed data path and the service path are implemented independently, separating the requirements for data transmission from those of device servicing The control path is physically separated from the main data path, so service operations do not directly affect high-speed data transmission The developed L2 combines up to 16 input streams from L1, each with an Ethernet rate of 10 Gb/s Hardware aggregation is performed in the FPGA using external DDR4 memory, after which the formed slice-blocks are transmitted to the Readout PC via PCI Express Gen3 x16 and Scatter-Gather DMA This organization moves data reception, buffering, sorting, and aggregation operations into the hardware part of the system and reduces the load on the software network subsystem of the Readout PC The modular design of the concentrators allows additional L1 and L2 units to be added as the number of readout channels and the corresponding data-processing requirements of the SPD experiment increase The hardware solutions use commercially available components and standard high-speed interfaces Experimental studies confirmed the operation of the developed components and the main data-transmission interfaces Acknowledgments This work was partially supported by Ministry of Science and Higher Education of the Russian Federation (project No.

Kai: FSWM-2025-0023) in the part of study of the L2 concentrator for readout chain The paper is prepared for submission to JINST The authors have provided contact emails boikov@jinr.ru, aleksandr.boikov@cern.ch The work was partially supported by Ministry of Science and Higher Education of the Russian Federation (project No.

Mira: FSWM-2025-0023) in the part of study of the L2 concentrator for readout chain The authors are L.

Kai: Afanasyeva A.

Title and authors: Kai: We're moving into segment two, and we're talking about the title and authors of this paper, "Readout chain of SPD DAQ system."

Mira: So the title itself is pretty descriptive; it just tells us exactly what part of the big experiment they are focusing on: the readout chain for the data acquisition system.

Lev: It sounds very focused on a specific piece of hardware rather than just the whole physics goal eight <ref:2610.16515#pg1>.

Kai: Yeah, and looking at the authors, we see people from JINR in Russia and Tomsk State University in Russia involved in this work one <ref:2610.16515#pg1>.

Mira: They are clearly working on integrating these hardware concepts into a large-scale experimental setup at NICA, which is a high-energy collider one <ref:2610.16515#pg1>.

Lev: I wonder if their expertise leans more toward the theoretical modeling of the detector components or the actual implementation of the DAQ hardware eight <ref:2610.16515#pg1>.

Kai: Based on what we’ve heard, it seems like they have a good mix, as Mira focuses on condensing matter theory and Lev is deep in quantum error correction for real hardware seven.

Mira: I think Lev's work with building L2 hubs for SPD at the NICA complex shows he's hands-on with the practical implementation challenges eight <ref:2610.16515#pg1>.

Lev: That’s right, and it highlights that bridging the gap between theoretical proposals and what actually runs on real hardware is a big part of his job eight <ref:2610.16515#pg1>.

Kai: So, in simple terms, this paper is about designing the physical structure for how data moves from the sensors to the computer in this specific experiment one <ref:2610.16515#pg1>.

Mira: It’s not about the physics results themselves, but making sure we can actually capture and organize those spin-related signals efficiently three <ref:2610.16515#pg1>.

Lev: So they are building a system that is robust enough to handle the expected interaction rates and channel counts one <ref:2610.16515#pg1>.

The paper's summary: Kai: Okay, so the paper summarizes their approach by describing this two-level structure: L1 and L2 concentrators three <ref:2610.16515#pg1>.

Mira: They explain that the L1 concentrator’s job is to be the interface between the front-end electronics and that next system level while distributing synchronous signals three <ref:2610.16515#pg1,the interface between the front-end electronics and>.

Lev: And they mention that they are using a White Rabbit based time-synchronization system to keep all those distributed elements perfectly timed three <ref:2610.16515#pg1>.

Kai: Then, the L2 concentrator’s main job is to aggregate data streams coming from several L1s before transmitting them to the readout PC four <ref:2610.16515#pg1,before transmitting them to the>.

Mira: And they stress that this aggregation happens in hardware, which is a key part of their design because it shifts the buffering and aggregation from a software network subsystem into the FPGA hardware four <ref:2610.16515#pg1>.

Lev: That’s what I mean, because when you're dealing with high data rates, having the hardware do the heavy lifting before it hits software is where you save critical time eight <ref:2610.16515#pg1>.

Kai: They also detail how they handle data transmission between L1 and L2 using Scatter-Gather DMA to move those slice-blocks to the readout PC via PCI Express Gen3 xsixteen four <ref:2610.16515#pg1>.

Mira: And that mechanism allows them to efficiently transfer data located in several separate DDR4 memory regions as part of a sequence of DMA operations four <ref:2610.16515#pg1>.

Lev: It’s a good way to manage the complexity, and it shows how they are managing the complexity by using specific hardware mechanisms instead of software libraries eight <ref:2610.16515#pg1>.

Kai: And they also include details on the control and monitoring path which is physically separated from the main data path three <ref:2610.16515#pg1>.

Mira: That separation is important because it ensures that service operations don't directly affect the high-speed data transmission flow four <ref:2610.16515#pg1>.

Lev: That independence helps guarantee stability, which is important when you are running complex systems at this scale, especially in a real experimental setting eight <ref:2610.16515#pg1>.

The paper's improvements: Kai: Now they discuss their suggested improvements, and the main focus there is on increasing scalability nine <ref:2610.16515#pg1>.

Mira: They propose that by keeping the principles of the hierarchy and interfaces between levels the same, you can achieve scaling by just adding more L1 and L2 concentrators nine <ref:2610.16515#pg1>.

Lev: So they are not proposing a complete overhaul of the design but rather an expansion, which is what makes it realistic for real hardware development eight <ref:2610.16515#pg1>.

Kai: They also explicitly mention that this modular design allows additional L1 and L2 units to be added as the number of readout channels increases three <ref:2610.16515#pg1>.

Mira: This means if you need more channels, you just add more concentrators without changing the core structure of the DAQ branch nine <ref:2610.16515#pg1>.

Lev: That’s a practical improvement because it avoids having to re-engineer everything just because the required channel count goes up significantly eight <ref:2610.16515#pg1>.

Kai: And they also stress that they moved reception, buffering, sorting, and aggregation operations entirely into hardware four <ref:2610.16515#pg1>.

Mira: That hardware implementation of those functions is what gives them the efficiency gains over a pure software approach four <ref:2610.16515#pg1>.

Lev: It’s the core idea—get it done in silicon where it needs to be done for performance reasons eight <ref:2610.16515#pg1>.

Conclusion: Kai: So, wrapping up this paper, the main implication is that they have created a hardware solution for the readout chain of the SPD DAQ system one <ref:2610.16515#pg1>.

Mira: It shows how essential it is to implement this in hardware for managing the high data rates and complexity inherent in such experiments four <ref:2610.16515#pg1>.

Lev: For me, it’s about showing that you can build something that actually runs on real hardware with the expected throughput of twenty GB/s one <ref:2610.16515#pg1>.

Kai: The modular design means they can keep adding L1 and L2 units as the experiment grows without changing the core structure three <ref:2610.16515#pg1>.

Mira: It also confirms through experimental studies that this entire chain, including all components, operates exactly how they designed it should seven.

Lev: It’s a solid piece of hardware that addresses the scaling needs while keeping things stable and manageable eight <ref:2610.16515#pg1>.

Joint Institute for Nuclear Research · National Research Tomsk State University

physics.ins-det

Submitted: 2026-10-12

Updated: 2026-10-29

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 75/100

The gist: The Spin Physics Detector (SPD) experiment at the NICA collider aims to study the spin structure of the nucleon and spin-related phenomena in interactions of polarized proton and deuteron beams The

Key concepts

L1 Concentrator: The first-level node in the readout chain, built on an Intel Cyclone 10 GX FPGA. It connects the Front End Electronics (FEE) to L2. Its job is to handle data processing, interface control with the FEE, and network communication with L2 using a copper differential interface.
L2 Aggregation Strategy: The second-level concentrator combines multiple data streams from several L1 units in hardware. This aggregation happens directly in the FPGA rather than via software. It uses Scatter-Gather DMA to efficiently move aggregated data, which is then sent to the Readout PC.
Scatter-Gather DMA: A technique used by L2 to transfer data from multiple separate DDR4 memory regions in a single sequence of operations. This allows the system to efficiently move large 'slice-blocks' of aggregated data to the Readout PC memory without requiring complex software buffering, boosting transmission efficiency.
Time Synchronization: A precise time synchronization system based on White Rabbit technology is used across all distributed elements. This ensures that every data block receives a defined 'time slice,' which is critical for accurately reconstructing events and studying spin-related physics phenomena.

Terminology

Core — 6× in this paper · explained in 2 episodes
Spin Physics Detector — 5× in this paper
Generator — 3× in this paper · explained in 2 episodes
Hierarchical Structure — 2× in this paper · explained in 3 episodes
Communication Overhead — 1× in this paper · explained in 2 episodes
Monolitic Active Pixel Sensors — 1× in this paper
Start of Frame — 1× in this paper
Start of Slice — 1× in this paper

Summary

The Spin Physics Detector (SPD) experiment at the NICA collider aims to study the spin structure of the nucleon and spin-related phenomena in interactions of polarized proton and deuteron beams The experimental conditions, including the high interaction rate and the large number of readout channels, require a triggerless data acquisition system with high throughput and a scalable architecture

DAQ System Architecture

The readout chain of the SPD DAQ system is organized into two hierarchical levels for receiving, aggregating, and transmitting data from the FEE to the computing system As shown in figure 1, data from the FEE are transmitted to L1 concentrators, where several input streams are combined into one The aggregated data are then transmitted to L2 concentrators, which combine streams from several L1 concentrators and forward them to the computing system through a PCIe interface using direct memory access (DMA) The general DAQ architecture for the SPD experiment was proposed in [2], including a hierarchical structure of concentrators, data-flow organization principles, and a White Rabbit-based time-synchronization system

L1 Concentrator Functionality

The first-level concentrator (L1) is an intermediate node of the readout chain that connects the FEE to the second-level concentrator (L2) The L1 concentrator is based on an Intel Cyclone 10 GX FPGA, which implements data processing and transmission, FEE interface control, interaction with the synchronization system, and network communication with L2 Communication between the L1 concentrator and the FEE is implemented using a copper differential interface The interface supports bidirectional communication between L1 and the FEE

L2 Aggregation Strategy

The second-level concentrator (L2) combines data streams from several first-level concentrators (L1) and transmits the aggregated data to the Readout PC L2 performs the aggregation of multiple input streams in hardware and forms the data stream for the computing system This organization shifts data reception, buffering, and aggregation from the software network subsystem to the FPGA hardware L2 uses Scatter-Gather DMA, which allows data located in several separate DDR4 memory regions to be transferred as part of a sequence of DMA operations

Performance and Scalability Metrics

The expected number of FEE-L1 links for different detector configurations is given in table 1 The minimum number of concentrators at each level is determined by the total number of FEE outputs Nout, where NL1 = Nout/8 and NL2 = NL1/16 The physical line rate of each optical channel is 10.3125 Gb/s, corresponding to a total physical line rate of 165 Gb/s for 16 channels The effective payload data rate for 16 channels varies from 87.6 Gb/s (11.0 GB/s) at the minimum payload size to 156.0 Gb/s (19.5 GB/s) at the maximum Ethernet payload size

Control and Monitoring Path

A separate Control Logic block handles control commands, configuration data, and monitoring information exchanged between the external control network and the L2, L1, and FEE nodes The highspeed data path and the service path are implemented independently, separating the requirements for data transmission from those of device servicing The control path is physically separated from the main data path, so service operations do not directly affect high-speed data transmission

Conclusion on Hardware Implementation

The developed L2 combines up to 16 input streams from L1, each with an Ethernet rate of 10 Gb/s Hardware aggregation is performed in the FPGA using external DDR4 memory, after which the formed slice-blocks are transmitted to the Readout PC via PCI Express Gen3 x16 and Scatter-Gather DMA This organization moves data reception, buffering, sorting, and aggregation operations into the hardware part of the system and reduces the load on the software network subsystem of the Readout PC The modular design of the concentrators allows additional L1 and L2 units to be added as the number of readout channels and the corresponding data-processing requirements of the SPD experiment increase The hardware solutions use commercially available components and standard high-speed interfaces Experimental studies confirmed the operation of the developed components and the main data-transmission interfaces

Acknowledgments

This work was partially supported by Ministry of Science and Higher Education of the Russian Federation (project No. FSWM-2025-0023) in the part of study of the L2 concentrator for readout chain The paper is prepared for submission to JINST The authors have provided contact emails boikov@jinr.ru, aleksandr.boikov@cern.ch The work was partially supported by Ministry of Science and Higher Education of the Russian Federation (project No. FSWM-2025-0023) in the part of study of the L2 concentrator for readout chain The authors are L. Afanasyeva A. Berngardtb A. Boikova V. Borshchb D. Erofeevb V. Frolova K. Gritsaya V. Olshevskya A. Kulikova S. Tereshchenkoa V. Tereshchenkoa The authors are from the Joint Institute for Nuclear Research, Dubna, Russia and The National Research Tomsk State University, Tomsk, Russia

References

[1] SPD Collaboration. Technical design report of the spin physics detector at NICA. Natural Science Review, 1, 2024 [3] White rabbit specification, 2025 [4] Federico Bertolucci et al. Level-1 data driver card of the atlas new small wheel. ATL-MUON-PROC, (2016-010), 2016 [8] A. Berngardt, D. Erofeev, and V. Borshch. Development of an L2 hub for the SPD experiment at the NICA accelerator complex. Izvestiya vysshikh uchebnykh zavedenii. Fizika, 67(12):85–90, 2024 [9] A. Berngardt, D. Erofeev, V. Borshch, and A. Boikov. Integration of the L2 concentrator into the SPD experiment at the NICA accelerator complex. Izvestiya vysshikh uchebnykh zavedenii. Fizika, 68(12):104–109, 2025 [10] Xilinx. UltraScale+ Devices Integrated Block for PCI Express. Xilinx, 2023 [7] Texas Instruments. LVDS Owner’s Manual, 2014 [5] Ieee standard for ethernet, 2002. Section 36.2.4.9, 8B/10B alignment comma characters [6] Elliott Cole. Performance of lvds with different cables. Technical Report SLLA053B, Texas Instruments, 2002 [7] Texas Instruments.

Improvements for AI systems

Direct hardware aggregation of small packets reduces CPU load by performing the aggregation directly in the FPGA before transmission to a Readout PC, shifting data reception, buffering, and aggregation from the software network subsystem to the FPGA hardware.
Enhanced system scalability allows for future expansion by enabling the transition from the first phase of the experiment to the full configuration is achieved by adding L1 and L2 concentrators without changing the principles of the hierarchy or the interfaces between its levels.
Improved data transmission efficiency is achieved through a Scatter-Gather DMA approach, which allows data located in several separate DDR4 memory regions to be transferred as part of a sequence of DMA operations, efficiently moving slice-blocks to the Readout PC memory.
Precise time synchronization ensures accurate event reconstruction by providing precise time synchronization of distributed elements, which defines time slices for each data block, crucial for studying spin-related phenomena.
Independent high-speed communication channels between L1 and L2 enable the system to meet scaling requirements by allowing the possibility of scaling the number of L2 concentrators without altering the fundamental DAQ branch design.