Design and hardware evaluation of the optical-link system for the ATLAS Liquid Argon Calorimeter Phase-II Upgrade

arXiv:2007.15854 · physics.ins-det, hep-ex · Submitted 2020-07-31 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Design and hardware evaluation of the optical-link system for the ATLAS Liquid Argon Calorimeter Phase-II Upgrade".

Mira: Here's the gist of the paper. An optical link system is being developed for the ATLAS Liquid Argon Calorimeter Phase-II upgrade because it is responsible for transmit the data of over 182 thousand detector channels from 1524 Front-End Boards (FEBs) through 26 optical fibers per FEB over 150 meters to the counting room and brings clocks, bunch crossing reset signals and slow control/monitoring signals back to the FEBs System Requirements for Upgrade The readout electronics of the ATLAS Liquid Argon Calorimeter will undergo a major upgrade (Phase-II) for the High-Luminosity Large Hadron Collider (HL-LHC) With no trigger selection, the data rate (225 Gbps per FEB) of the on-detector readout electronics after upgrade will be more than 100 times higher than that of the current readout electronics (1.6 Gbps per FEB) The distance from the detector to the counting room is about 150 meters To meet the upgrade demands of the ATLAS LAr Calorimeter, a radiation-tolerant optical link system is being developed and a printed circuit board (PCB) called the link demonstrator board has been designed Any component used in the on-detector electronics must tolerate at least a Total Ionizing Dose (TID) of 2.26 kGy, a Nonionizing Energy Loss (NIEL) of 4.9´1013 1-MeV-equivalent neutrons/cm2, and a Single-Event Effects (SEE) of 7.7´1012 hadrons/cm2 (including safety factors), respectively, for a total luminosity of 4000 fb-1 Optical Link Architecture The optical link system is based on the Low-Power GigaBit Transceivers (lpGBTs) and the Versatile optical Transceiver (VTRx+) modules, which both are being developed for the High-Luminosity LHC upgrade The lpGBT on the first demonstrator board (Demo1) operates in the transceiver mode and is responsible for control links, while the lpGBT on the second demonstrator (Demo2) operates in the transmitter mode and is responsible for the data link Two FPGA development kits (BE-FPGA1 and BE-FPGA2, Part No.

Kai: KC705 produced by Xilinx) emulate the off-detector FPGAs The lpGBT on Demo1 recovers a clock for the lpGBT on Demo2 A clock generator (Part number SI5338EVB from Silicon Laboratory) provides the clocks for the two off-detector FPGAs Data Transmission Design Each ADC takes 8 analog signals, half in low gain and the other half in high gain, from the preamplifiers The digitized signals of ADCs are serialized into 640 Mbps and output to lpGBTs in the format of CERNSpecified Low-Voltage Signaling (CLVS) format Each lpGBT acquires the data from 14 electrical ports (e-ports) at 640 Mbps, encodes the data by adding a frame header, an Internal Control (IC) field, an External Control (EC) field, and Forward Error Correction (FEC) code, and serializes the encoded data into a serial sequence at 10.24 Gbps The frame channel and the 8 normal data channels of the first ADC go to the first lpGBT Data Evaluation Results The uplink data were checked in BE-FPGA2 using a logic analyzer tool without any error for more than 15 minutes The bit error rate (BER) is estimated to be 1´10-12, the BER specification of lpGBT/VTRx+ and the ATLAS liquid argon calorimeter, with a confidence level of 99.99% The downlink data were received without any error for more than 15 minutes, corresponding to the BER of 1´10-12 with a confidence level of 90.0% Clock and Control Distribution Each preamplifier needs a 40 MHz clock to generate an internal clock for its InterIntegrated Circuit (I2C) functional part, while each ADC requires a 40 MHz clock for sampling and a 640 MHz clock for output data serialization The lpGBT on Demo1 recovers a recovered 40 MHz clock from the lpGBT on Demo1 and the regenerated 40 MHz and 640 MHz clocks on Demo2 The measured random jitter meets the system requirements of 5 ps (RMS) Configuration Flexibility The preamplifiers, ADCs, lpGBTs, and VTRx+ all have an I2C interface for remote configuration Each control lpGBT, which has three I2C controllers (M0, M1, and M2), is responsible for half FEB The lpGBT on Demo1 can be controlled remotely through its IC channel The verification of an lpGBT configured via the EC channel is still ongoing Power Dissipation Concerns The power dissipation of the optical link system (24 lpGBTs and 8 VTRx+ modules) on every FEB is estimated to be up to 20 W If we assume the conversion efficiency of 85% in DC-DC converters and a voltage drop of 1 V in Low Drop Out (LDO) regulators, the optical link system on each FEB consumes about 42 W, over 1/3 of the power consumption budget (110 W) of the whole FEB The possibility of eliminating the LDO regulators and minimizing the noise of DC-DC converters with a multiplephase synchronous operation will be explored in the prototype link board Future Design Directions A vertical slice of FEB called the prototype link board, which includes 24 lpGBTs and 8 VTRx+ modules, will be designed The prototype link board will be tested with the same FPGAs No redundancy is considered in the link demonstrator board except the crossover power cycle and reset of control lpGBTs Redundant distribution of clocks, BCRs, and I2C configuration will be investigated in the prototype link board The prototype link board will also address the system-level issues such as trace/fiber routing scheme, mechanical support, power dissipation, and cooling BCR Distribution Details Each control lpGBT has 16 transmitter e-ports and each e-port provides a BCR signal for an ADC By changing the data transmitted out of the e-port, the phase of the BCR signal can be adjusted in steps of 12.5 ns We also confirm that the phase of the BCR signal can be shifted in steps of 12.5 ns The pulse width of the measured BCR signal is 25 ns and the period is 89.1 µs, or 3564 cycles of the LHC bunch crossing clock Control Link Monitoring First, we can turn on or off all power supply modules and reset ASICs Second, we monitor the temperatures at various positions Third, we monitor all power supply voltages via the embedded ADC of lpGBTs The input signal range of the embedded ADC of the lpGBT is from 0 V to 1 V When a power supply voltage to be monitored is larger than 1 V (for example, 2.5 V for preamplifiers), a resistor-division network must be used The rate to monitor temperatures and voltages is at the order of milliseconds because the conversion start and the poll of conversion completion are controlled in software An optical link system is being produced for the ATLAS LAr Calorimeter Phase-II upgrade A link demonstrator board is designed and major functions of the optical link system have been evaluated, though extensive tests are still ongoing The prototype link board will be tested with the same FPGAs No redundancy is considered in the link demonstrator board except the crossover power cycle and reset of control lpGBTs Redundant distribution of clocks, BCRs, and I2C configuration will be investigated in the prototype link board The prototype link board will also address the system-level issues such as trace/fiber routing scheme, mechanical support, power dissipation, and cooling The power dissipation of the optical link system (24 lpGBTs and 8 VTRx+ modules) on every FEB is estimated to be up to 20 W If we assume the conversion efficiency of 85% in DC-DC converters and a voltage drop of 1 V in Low Drop Out (LDO) regulators, the optical link system on each FEB consumes about 42 W, over 1/3 of the power consumption budget (110 W) of the whole FEB The possibility of eliminating the LDO regulators and minimizing the noise of DC-DC converters with a multiplephase synchronous operation will be explored in the prototype link board REFERENCE 2007.15854 Page 1 --- Design and hardware evaluation of the optical-link system for the ATLAS Liquid Argon Calorimeter Phase-II Upgrade B.

Mira: Deng, C.

Kai: Liu, D.

Title and authors: Kai: The title is very direct: "Design and hardware evaluation of the optical-link system for the ATLAS Liquid Argon Calorimeter Phase-II Upgrade". It tells you exactly what they built and why they built it.

Mira: And the authors are a team from different places, which is always interesting when you're looking at something this large-scale. You have people from electronic engineering, physics, and even some institutes in Taiwan and China.

Lev: It shows that this kind of massive hardware development requires a lot of cross-disciplinary expertise to get it right for the real world.

Kai: The implication here is that they're not just sketching ideas; they've actually designed and evaluated a printed circuit board called the link demonstrator board.

Mira: That evaluation is key because you need to know if something can survive the harsh radiation environment of a future collider, which is mentioned right there in the introduction.

Lev: They are specifically looking at components that have to handle a total ionizing dose of two point two six kilograys and a single event effect level of seven point seven times ten twelve hadrons per cubic centimeter for that four thousand femtobarns luminosity <ref:2007.15854#pg2>.

Kai: So, they’re setting very tough requirements right from the start on the hardware they are designing.

The paper's summary: Mira: The summary points out that the data rate after this upgrade is more than one hundred times higher than what's currently there, going from one point six gigabits per FEB to over two hundred and twenty-five gigabits per FEB <ref:2007.15854#pg2>.

Kai: That huge increase in data volume is what drives everything else—the need for a radiation-tolerant optical link system spanning about one hundred fifty meters <ref:2007.15854#pg1>.

Lev: The paper explains the architecture: it uses low-power GigaBit Transceivers, or lpGBT, and Versatile optical Transceiver modules, or VTRx plus modules, which are both being developed for the High-Luminosity LHC upgrade.

Mira: It breaks down how the control links and data links are separated; one lpGBT handles control on Demo one, while another handles the data link on Demo two.

Kai: They also detail a specific data transmission design where each ADC serializes its eight signals into six hundred forty megabits per second in a CERNSpecified Low-Voltage Signaling format.

Lev: And they show how the lpGBT acquires this at six hundred forty megabits per second on fourteen electrical ports, encodes it with headers and FEC, and then sends it out at ten point two four gigabits per second.

The paper's improvements: Kai: They focused a lot on the data transmission design, showing the mapping from ADCs to lpGBTs which looks like a complex flow of channels.

Mira: They introduced a frame channel in each ADC with fields like PARITY and BCID, which is used to recover word boundaries after serialization because those are lost.

Lev: This recovery mechanism relies on the off-detector FPGA to figure out the full BCID count, which is crucial for getting all the data back correctly.

Kai: The data evaluation results are pretty strong: they checked uplink and downlink data in BE-FPGA two without any errors for over fifteen minutes.

Mira: And they estimated a bit error rate of one to ten to the twelve, with a confidence level of ninety-nine point nine nine percent for the uplink.

Lev: For the downlink, they got a bit error rate of one to ten to the twelve with a confidence level of ninety percent after fifteen minutes. That suggests high reliability for what they're doing there.

Conclusion: Kai: To wrap up, this paper presents a design and evaluation board for the ATLAS Liquid Argon Calorimeter Phase-II upgrade's optical link system, showing it works with a bit error rate of one to ten to the twelve in tests.

Mira: The paper points forward by detailing improvements they plan to investigate on a future prototype link board, specifically focusing on redundancy for clocks and control signals.

Lev: They also look at the power dissipation challenge, estimating that the optical system on every FEB could use up to twenty watts, which is over one third of the hundred ten watt power budget for the whole FEB.

Kai: The idea there is to explore eliminating those low drop out regulators and using a multiple phase synchronous operation in the prototype board to minimize noise.

Mira: So, while they've proven the core data link works with high reliability, their next step is tackling the physical integration issues like routing and cooling on a full-scale prototype.

Lev: And for my part, I think testing that redundancy for clocks and I2c configuration in the prototype board is really important because you can't afford any single point of failure when you're running at those data rates.

Kai: So, to summarize, they built a demonstrator board that proves the core link functions reliably under specified conditions, but the real work is now moving to a more complete prototype design.

B. Deng, C. Liu, D. Gong, X. Huang, S. Hou, T. Liu, H. Sun, L Zhang W., Änd J Yec

Department of Electronic Information Engineering, Hubei Polytechnic University, Huangshi, Hubei 435003, P.R. China · Department of Physics, Southern Methodist University, Dallas, TX 75275, U.S.A · Department of Physics, Central China Normal University, Wuhan, Hubei 430079, P.R. China · Institute of Physics, Academia Sinica, Nangang, Taipei 11529 · Zhongyi Environmental Protection College, Yixing Environmental Protection Science and Technology Industrial Park, Yixing Jiangsu 214200

physics.ins-det, hep-ex

Submitted: 2020-07-31

Updated: 2020-07-31

Comments: 12 pages, 8 figures

Journal ref: Nuclear Inst. and Methods in Physics Research, A 981 (2020) 164495

DOI: 10.1016/j.nima.2020.164495

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 80/100

The gist: An optical link system is being developed for the ATLAS Liquid Argon Calorimeter Phase-II upgrade because it is responsible for transmit the data of over 182 thousand detector channels from 1524…

Key concepts

lpGBT
Low-Power GigaBit Transceivers are the core components used for the optical link. They handle both control links and data links, encoding serialized data into a high-speed serial sequence for transmission between detector boards.
VTRx+
Versatile optical Transceiver modules are also used in the system. These modules work alongside lpGBT to establish the physical optical connections necessary to transmit the digitized signals from the detectors to the counting room.
BCR Signal
Bunch Crossing Reset (BCR) signals are used for precise timing and synchronization. By changing data transmitted through these ports, researchers can adjust the phase of this signal in 12.5 ns steps, which is critical for triggering events at the LHC.
Power Dissipation Budget
The optical link system consumes up to 42W per Front-End Board (FEB), which is over one-third of the FEB's total power budget. Future design goals include reducing this consumption by exploring multi-phase synchronous operation and eliminating Low Drop Out regulators.

Terminology

Summary

An optical link system is being developed for the ATLAS Liquid Argon Calorimeter Phase-II upgrade because it is responsible for transmit the data of over 182 thousand detector channels from 1524 Front-End Boards (FEBs) through 26 optical fibers per FEB over 150 meters to the counting room and brings clocks, bunch crossing reset signals and slow control/monitoring signals back to the FEBs

System Requirements for Upgrade

The readout electronics of the ATLAS Liquid Argon Calorimeter will undergo a major upgrade (Phase-II) for the High-Luminosity Large Hadron Collider (HL-LHC) With no trigger selection, the data rate (225 Gbps per FEB) of the on-detector readout electronics after upgrade will be more than 100 times higher than that of the current readout electronics (1.6 Gbps per FEB) The distance from the detector to the counting room is about 150 meters To meet the upgrade demands of the ATLAS LAr Calorimeter, a radiation-tolerant optical link system is being developed and a printed circuit board (PCB) called the link demonstrator board has been designed Any component used in the on-detector electronics must tolerate at least a Total Ionizing Dose (TID) of 2. p. 126 kGy, a Nonionizing Energy Loss (NIEL) of 4. p. 1 p. 19´1013 1-MeV-equivalent neutrons/cm2, and a Single-Event Effects (SEE) of 7. p. 1 p. 17´1012 hadrons/cm2 (including safety factors), respectively, for a total luminosity of 4000 fb-1

Optical Link Architecture

The optical link system is based on the Low-Power GigaBit Transceivers (lpGBTs) and the Versatile optical Transceiver (VTRx+) modules, which both are being developed for the High-Luminosity LHC upgrade The lpGBT on the first demonstrator board (Demo1) operates in the transceiver mode and is responsible for control links, while the lpGBT on the second demonstrator (Demo2) operates in the transmitter mode and is responsible for the data link Two FPGA development kits (BE-FPGA1 and BE-FPGA2, Part No. KC705 produced by Xilinx) emulate the off-detector FPGAs The lpGBT on Demo1 recovers a clock for the lpGBT on Demo2 A clock generator (Part number SI5338EVB from Silicon Laboratory) provides the clocks for the two off-detector FPGAs

Data Transmission Design

Each ADC takes 8 analog signals, half in low gain and the other half in high gain, from the preamplifiers The digitized signals of ADCs are serialized into 640 Mbps and output to lpGBTs in the format of CERNSpecified Low-Voltage Signaling (CLVS) format Each lpGBT acquires the data from 14 electrical ports (e-ports) at 640 Mbps, encodes the data by adding a frame header, an Internal Control (IC) field, an External Control (EC) field, and Forward Error Correction (FEC) code, and serializes the encoded data into a serial sequence at 10.24 Gbps The frame channel and the 8 normal data channels of the first ADC go to the first lpGBT

Data Evaluation Results

The uplink data were checked in BE-FPGA2 using a logic analyzer tool without any error for more than 15 minutes The bit error rate (BER) is estimated to be 1´10-12, the BER specification of lpGBT/VTRx+ and the ATLAS liquid argon calorimeter, with a confidence level of 99.99% The downlink data were received without any error for more than 15 minutes, corresponding to the BER of 1´10-12 with a confidence level of 90. p. 50%

Clock and Control Distribution

Each preamplifier needs a 40 MHz clock to generate an internal clock for its InterIntegrated Circuit (I2C) functional part, while each ADC requires a 40 MHz clock for sampling and a 640 MHz clock for output data serialization The lpGBT on Demo1 recovers a recovered 40 MHz clock from the lpGBT on Demo1 and the regenerated 40 MHz and 640 MHz clocks on Demo2 The measured random jitter meets the system requirements of 5 ps (RMS)

Configuration Flexibility

The preamplifiers, ADCs, lpGBTs, and VTRx+ all have an I2C interface for remote configuration Each control lpGBT, which has three I2C controllers (M0, M1, and M2), is responsible for half FEB The lpGBT on Demo1 can be controlled remotely through its IC channel The verification of an lpGBT configured via the EC channel is still ongoing

Power Dissipation Concerns

The power dissipation of the optical link system (24 lpGBTs and 8 VTRx+ modules) on every FEB is estimated to be up to 20 W If we assume the conversion efficiency of 85% in DC-DC converters and a voltage drop of 1 V in Low Drop Out (LDO) regulators, the optical link system on each FEB consumes about 42 W, over 1/3 of the power consumption budget (110 W) of the whole FEB The possibility of eliminating the LDO regulators and minimizing the noise of DC-DC converters with a multiplephase synchronous operation will be explored in the prototype link board

Future Design Directions

A vertical slice of FEB called the prototype link board, which includes 24 lpGBTs and 8 VTRx+ modules, will be designed The prototype link board will be tested with the same FPGAs No redundancy is considered in the link demonstrator board except the crossover power cycle and reset of control lpGBTs Redundant distribution of clocks, BCRs, and I2C configuration will be investigated in the prototype link board The prototype link board will also address the system-level issues such as trace/fiber routing scheme, mechanical support, power dissipation, and cooling

BCR Distribution Details

Each control lpGBT has 16 transmitter e-ports and each e-port provides a BCR signal for an ADC By changing the data transmitted out of the e-port, the phase of the BCR signal can be adjusted in steps of 12.5 ns We also confirm that the phase of the BCR signal can be shifted in steps of 12. p. 85 ns The pulse width of the measured BCR signal is 25 ns and the period is 89. p. 81 µs, or 3564 cycles of the LHC bunch crossing clock

Control Link Monitoring

First, we can turn on or off all power supply modules and reset ASICs Second, we monitor the temperatures at various positions Third, we monitor all power supply voltages via the embedded ADC of lpGBTs The input signal range of the embedded ADC of the lpGBT is from 0 V to 1 V When a power supply voltage to be monitored is larger than 1 V (for example, 2.5 V for preamplifiers), a resistor-division network must be used The rate to monitor temperatures and voltages is at the order of milliseconds because the conversion start and the poll of conversion completion are controlled in software

Conclusion Summary

An optical link system is being produced for the ATLAS LAr Calorimeter Phase-II upgrade A link demonstrator board is designed and major functions of the optical link system have been evaluated, though extensive tests are still ongoing The prototype link board will be tested with the same FPGAs No redundancy is considered in the link demonstrator board except the crossover power cycle and reset of control lpGBTs Redundant distribution of clocks, BCRs, and I2C configuration will be investigated in the prototype link board The prototype link board will also address the system-level issues such as trace/fiber routing scheme, mechanical support, power dissipation, and cooling The power dissipation of the optical link system (24 lpGBTs and 8 VTRx+ modules) on every FEB is estimated to be up to 20 W If we assume the conversion efficiency of 85% in DC-DC converters and a voltage drop of 1 V in Low Drop Out (LDO) regulators, the optical link system on each FEB consumes about 42 W, over 1/3 of the power consumption budget (110 W) of the whole FEB The possibility of eliminating the LDO regulators and minimizing the noise of DC-DC converters with a multiplephase synchronous operation will be explored in the prototype link board

REFERENCE

2007.15854

--- Page 1 ---

Design and hardware evaluation of the optical-link system for the ATLAS Liquid Argon Calorimeter Phase-II Upgrade

B. Deng, C. Liu, D. Gong, X. Huang, S. Hou, T. Liu, H. Sun, L Zhang W Zhang and J Yec a Department of Electronic Information Engineering, Hubei Polytechnic University, Huangshi 435003 P. p. 0R. China b Department of Physics Southern Methodist University Dallas TX 75275 U. p. 0S.A c Department of Physics Central China Normal University Wuhan Hubei 430079 P. p. 0R. China d Institute of Physics Academia Sinica Nangang Taipei 11529 Taiwan E-mail: tliu@mail. p. 0smu.edu

ABSTRACT: An optical link system is being developed for the ATLAS Liquid Argon Calorimeter Phase-II upgrade The optical link system is responsible for transmit the data of over 182 thousand detector channels from 1524 Front-End Boards (FEBs) through 26 optical fibers per FEB over 150 meters to the counting room and brings clocks, bunch crossing reset signals and slow control/monitoring signals back to the FEBs The optical link system is based on the Low-Power GigaBit Transceivers (lpGBTs) and the Versatile optical Transceiver (VTRx+) modules which both are being developed for the High-Luminosity LHC upgrade An evaluation board is designed and the major functions of the optical link system are being evaluated The design of the optical link system and the evaluation of major functions are presented in the paper

KEYWORDS: Optical-link; Front-end electronics for detector readout; Data acquisition circuits.

1 Corresponding author. p. 0

--- Page 2 ---

  1. Introduction

The readout electronics of the ATLAS Liquid Argon Calorimeter will undergo a major upgrade (Phase-II) for the High-Luminosity Large Hadron Collider (HL-LHC) [1] In the proposed upgrade, the data of over 182 thousand detector channels are transmitted from the 1524 Front-End boards (FEBs) to the off-detector counting room without any trigger selection [2] With no trigger selection, the data rate (225 Gbps per FEB) of the on-detector readout electronics after upgrade will be more than 100 times higher than that of the current readout electronics (1.6 Gbps per FEB) [3] The distance from the detector to the counting room is about 150 meters Based on the data rate and the distance, optical fiber links are chosen for data transmission The on-detector electronics operate in a harsh radiation environment Any component used in the on-detector electronics must tolerate at least a Total Ionizing Dose (TID) of 2. p. 126 kGy, a Nonionizing Energy Loss (NIEL) of 4.9´1013 1-MeV-equivalent neutrons/cm2, and a Single-Event Effects (SEE) of 7.7´1012 hadrons/cm2 (including safety factors), respectively, for a total luminosity of 4000 fb-1 To meet the upgrade demands of the ATLAS LAr Calorimeter, a radiation-tolerant optical link system is being developed and a printed circuit board (PCB) called the link demonstrator board has been designed

--- Page 3 ---

  1. Overall architecture of the optical link system and the demonstrator system

Figure 1 is the block diagram of the overall architecture of a FEB In the figure, 128 analog channels are amplified and shaped in 32 4-channel Pre-Amplifiers (PAs, including shaping amplifiers) [4] and then are digitized in 32 8-channel Analog-to-Digital Converters (ADCs) [5] Each analog channel is digitized with two different gains simultaneously in order to achieve a 16-bit dynamic range The digitized data are transmitted from the detector to the off-detector counting room through optical links These uplinks are called data links We also need control links to provide clock and control/monitoring support Each FEB has 22 simplex data links and 2 duplex control links Optical links are based on lpGBTs [6] and VTRx+ modules [7] which both are being developed for the High-Luminosity LHC upgrade In this paper, the lpGBT responsible for data links will be called data lpGBTs, and the lpGBT responsible for control links will be called control lpGBTs

--- Page 4 ---

Figure 2(a) is the block diagram of the link demonstrator system with two demonstrator boards and corresponding FPGAs Figure 1. Block diagram of the overall architecture of FEB (a) (b)

Figure 2. Block diagram (a) and photograph (b) of the optical link demonstrator system

  1. Design and evaluation of data transmission

3.1 Design of data transmission The interface from ADCs to lpGBTs is shown in Figure 3(a) Each ADC takes 8 analog signals, half in low gain and the other half in high gain, from the preamplifiers The digitized signals of ADCs are serialized into 640 Mbps and output to lpGBTs in the format of CERNSpecified Low-Voltage Signaling (CLVS) format [9] It should be noted that lpGBT have intrinsic bit alignment mechanics to ensure that lpGBT do not sample the data on bit boundaries However, word boundaries are lost after serialization Therefore, we must introduce work boundary recovery mechanics Each ADC has two extra output channels call the frame channels The word in a frame channel is defined as 1, PARITY, FIRSTEVENT, BCID[4:0], 00000000 We use the frame channel to recover the word boundary and the full BCID count in the off-detector FPGA Each lpGBT acquires the data from 14 electrical ports (e-ports) at 640 Mbps, encodes the data by adding a frame header, an Internal Control (IC) field, an External Control (EC) field, and Forward Error Correction (FEC) code, and serializes the encoded data into a serial sequence at 10. p. 324 Gbps

--- Page 5 ---

Figure 3. Mapping from ADCs to lpGBTs (a) and data flow of ½ FEB (b)

lpGBT lpGBT 10.24 Gbps Eport[13:0] Eport[13:0] ADC ADC frame[0] Ch[4:1] Ch[8:5] frame[1] lpGBT 2 lpGBT 1 ADC 2 ADC 3 frame[0] Ch[4:1] Ch[8:5] frame[1] lpGBT 3

lpGBT 6

lpGBT 5

ADC 8 ADC 9 ADC 7 frame[0] Ch[4:1] Ch[8:5] frame[1] lpGBT 8

lpGBT 7 lpGBT 10 ADC 11 ADC 2 frame[0] Ch[4:1] Ch[8:5] frame[1] lpGBT 9

lpGBT 4

lpGBT 3

ADC 14 ADC 5 ADC 6 frame[0] Ch[4:1] Ch[8:5] frame[1] lpGBT 6

--- Page 6 ---

Figure 3(b) is the data flow of half FEB In the figure, lpGBTs 1-11 are for data links and lpGBT 12 is for control links Each half FEB has 16 preamplifiers, 16 ADCs, 12 lpGBTs, 4 VTRx+ modules, and 13 optical fibers

--- Page 7 ---

Figure 5. Block diagram of clock distribution on a half FEB

lpGBT the lpGBT operating in the transceiver mode recovers 11 MHz clocks from the received downlink data for lpGBTs 1-11 Each data lpGBT operating in the transmitter includes a Phase-Locked Loop (PLL) and regenerates 40 MHz and 640 MHz clocks for preamplifiers and ADCs The random jitter of the regenerated 40 MHz clocks is about 2.2 ps (RMS) In our application, the 40 MHz clock is used in ADCs as the sampling clock and sensitive to jitter The measured random jitter meets the system requirements of 5 ps (RMS) Significant periodic jitter components of about 23 ps (peak-to-peak) are observed in the regenerated 640 MHz clock

--- Page 8 ---

Figure 6. Block diagram of BCR distribution

ADC lpGBT FPGA recovers the 12-bit BCID count based on the five-bit count and the FIRSTEVENT flag To operate the FIRSTEVENT flag and the BCID counter, each ADC requires a BCR signal to set its FIRSTEVENT flag and to reset its internal counter Each control lpGBT has 16 transmitter e-ports and each e-port provides a BCR signal for an ADC By changing the data transmitted out of the e-port, the phase of the BCR signal can be adjusted in steps of 12.5 ns We have evaluated a BCR distribution scheme on the link demonstrator board Figure 6(b) displays the measured waveform of a BCR signal in purple related to a 40 MHz clock as the trigger in green The pulse width of the measured BCR signal is 25 ns and the period is 89. p. 81 µs, or 3564 cycles of the LHC bunch crossing clock We also confirm that the phase of the BCR signal can be shifted in steps of 12. p. 8 p. 85 ns

--- Page 9 ---

We have evaluated a BCR distribution scheme on the link demonstrator board Figure 6(b) displays the measured waveform of a BCR signal in purple related to a 40 MHz clock as the trigger in green The pulse width of the measured BCR signal is 25 ns and the period is 89.1 µs, or 3564 cycles of the LHC bunch crossing clock We also confirm that the phase of the BCR signal can be shifted in steps of 12.5 ns

REFERENCE

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Vertical slice prototype link board design will enable study of redundancy design of control links without considering them in current demonstrator boards except for crossover power cycle and reset of control lpGBTs. This will allow investigation into redundant distribution of clocks, BCRs, and I2C configuration in the next version.

  2. The prototype link board will address system-level issues such as trace/fiber routing scheme, mechanical support, power dissipation, and cooling. This allows for a comprehensive study of physical integration challenges for the full system.

  3. Power dissipation estimation shows that the optical link system (24 lpGBTs and 8 VTRx+ modules) on every FEB is estimated to be up to 20 W, leading to a total consumption of about 42 W, over 1/3 of the power consumption budget (110 W) of the whole FEB. The prototype board will explore the possibility of eliminating the LDO regulators and minimizing the noise of DC-DC converters with a multiplephase synchronous operation.

  4. The system can achieve a bit error rate (BER) estimated at 1e-12, the BER specification of lpGBT/VTRx+ and the ATLAS liquid argon calorimeter, with a confidence level of 99.99% for uplink data and 1e-12 with a confidence level of 90.0% for downlink data, indicating high reliability for detector readout.

  5. The system can monitor all power supply voltages via the embedded ADC of lpGBT's, using a scheme where the input signal range of the embedded ADC is from 0 V to 1 V and employing a resistor-division network when monitoring voltages larger than 1 V, allowing for comprehensive power monitoring.

Related papers