The EDD Radio Astronomy Backend Framework
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.
Vera: We're looking at the paper: "The EDD Radio Astronomy Backend Framework".
Jocelyn: Here's the gist of the paper. The EDD Radio Astronomy Backend Framework presents a software-defined architecture designed to create reusable, scalable real-time radio astronomy backends on commodity hardware by separating common core services from observing mode plugins Framework Design and Architecture The EDD backend is structured as a modular, distributed system consisting of loosely coupled processing and control elements The system is composed of services, which are persistent components providing functionality like control, interfacing, and monitoring, and pipelines, which are ephemeral components executing data-processing or resource-handling tasks Pipelines are organized into observing modes by connecting them through well-defined data streams forming a directed acyclic graph (DAG) specified by a provision description The operation of the system at each site is coordinated by a Master Controller, which manages the lifecycle of pipelines and services and enforces the system state model Operational State Model All EDD pipelines adhere to a common state model that governs their lifecycle and operational control This model is organized as a hierarchy of three control loops corresponding to distinct reconfiguration timescales in radio astronomical observations The pipeline transitions between states such as unprovisioned, idle, configured, and measuring based on explicit commands from the Master Controller Specific commands like capture start transition pipelines into operational states depending on their class Failure modes include an error state, a stalled state, and a panic state indicating recovery requirements Data Flow and Communication Communication within the system is separated into control and data channels Control communication relies on imperative command messages issued by the Master Controller to pipelines defining state transitions Dataflow is realized through data streams connecting pipelines in the processing graph, where these streams define the interfaces and specify how data is exchanged The architecture is intentionally agnostic with respect to underlying data transport mechanisms, allowing pipelines to exchange data using various methods depending on deployment needs The primary method utilized is IP multicast over Ethernet for efficient fan-out and decoupling producers from consumers Observational Capabilities and Deployments The framework currently supports a broad range of observing modes through its plugin architecture, including spectroscopy, pulsar timing and searching, baseband recording, VLBI, correlation, and beamforming Representative modes include spectroscopy and spectropolarimetry which uses GPU-accelerated channelisation for simultaneous intensity and polarisation product recovery Pulsar timing supports two ingest paths depending on data rate, utilizing a CASPER-based FPGA polyphase filterbank plugin for channelisation in higher data rate scenarios The framework has been deployed at four sites: the Effelsberg 100-m radio telescope (EDGAR), SKAMPI, the Thai National Radio Telescope (TNRT), and the ARGOS interferometric prototype array Performance and Comparative Results When coupled with in-receiver digitisation, the EDD backend at Effelsberg provides ∼ 30% improvement in pulsar time-of-arrival estimation compared with previous instruments when using the same observing bandwidth and integration time For the P217mm receiver comparison, EDD showed that measured signal-to-noise ratios increase by 20–110% and corresponding pulse time-ofarrival uncertainties are reduced by approximately 20–40% compared to the legacy PSRIX backend This improvement is primarily attributed to digitising close to the receiver, which avoids signal degradation over the analogue transmission path The framework demonstrates scalability across different hardware architectures, including disaggregated, converged, and hyper-converged configurations at various deployments Implementation and Maintenance Deployment and provisioning of observation modes are managed using Ansible for configuration management and application deployment The EDD core and plugins are packaged as Ansible collections, enabling a configuration-as-code approach where the full system state is version-controlled Services run in isolated Docker containers, providing decoupled environments that integrate into diverse computing setups without imposing constraints on host distributions Monitoring and logging are handled through a combination of in-memory and time-series databases visualized via Grafana, with logs collected centrally from Docker output using Loki Dependency management is addressed through containerisation, fixed software suites for the core and plugins, and building against fixed Linux distributions to ensure compatibility across the stack The EDD backend provides a reusable open-source framework for developing and operating real-time radio astronomy backends by separating common services from site-specific configuration and observing-mode plugins It successfully supports operations on both single dishes and interferometers while accommodating substantially different hardware, network, and control-system environments The framework serves as a community resource for developing maintainable real-time backends for broadband radio astronomy instruments Pruning redundant reasoning tokens The EDD backend addresses an ongoing challenge in modern radio astronomy: wide-band, directly digitised receivers increasingly require backend systems that can be reconfigured, maintained, and extended without rebuilding the instrument software for each observing mode or telescope How the training signal is built The EDD framework builds its capabilities through a core framework providing essential services like the Master Controller and interface services, while plugins supply processing pipelines for specific observing modes These pipelines are composed into observing modes by connecting them via data streams forming a DAG that ensures deterministic configuration and execution Where the gains come from The performance gains observed in pulsar timing estimations stem from digitising close to the receiver, which avoids signal degradation over the analogue transmission path The framework's success lies in its ability to support multiple science cases commensally on shared infrastructure by separating common services and observing-mode plugins Deployment and Provisioning of Observation Modes The EDD backend is deployed using Ansible, which automates software provisioning, configuration management, and application deployment via playbooks Installation involves pulling container images from a central registry and generating configuration files from an inventory to achieve reproducibility through a configuration-as-code approach Observation modes are provisioned by the user using commands sent to the Master Controller, which executes Ansible playbooks to launch the required pipeline and service containers Control and Protocols Communication between the Master Controller and pipelines uses the KAroo Telescope Communication Protocol (KATCP) for imperative command messages defining state transitions Infrastructure-level operations are managed by Ansible over SSH, while KATCP provides the runtime control layer through which the Master Controller orchestrates pipeline behavior during observations Data Transport The EDD framework is agnostic to data transport mechanisms, primarily utilizing IP multicast over Ethernet for efficient publish-subscribe data exchange between pipelines The SPEAD protocol encodes self-describing items into UDP packet streams, supporting complex multidimensional data types suitable for dynamic pipeline topologies High-performance transmission is achieved using tools like eddio, which leverages DPDK for low-level packet handling to achieve high line-rate performance Monitoring and Logging For operational monitoring, every pipeline exposes default metrics and pipeline-specific data via KATCP sensors collected by sidecar monitoring services Collected data is stored in an in-memory database for low-latency access to current state and a time-series database for high-resolution scalar metrics like packet loss rates Aggregated data from both systems is visualized using Grafana, while logs are collected from the Docker daemon and ingested into Loki for centralized querying Observational Modes The EDD backend supports various modes such as spectroscopy and spectropolarimetry, which employs GPU-accelerated channelisation and cross-polarisation coherence detection Pulsar timing and searching support unchannelised or channelised dual-polarisation voltage streams, with a CASPER-based FPGA plugin handling subband distribution for higher data rates VLBI capabilities involve a GPU-accelerated digital down-converter (DDC) pipeline that extracts arbitrary sidebands and packages them in standard VDIF format Acknowledgements Part of the work was based on observations with the 100-m telescope of the MPIfR at Effelsberg The SKAMPI prototype telescope is a facility of the Max Planck Society and was established with assistance from SARAO This research received funding from the MPG and the European Union’s Horizon Europe research and innovation programme under grant agreements No 101093934 (RADIOBLOCKS) and 101094354 (ARGOS-CDS) References Barr, E.
Vera: D.
Jocelyn: 2017, Proceedings of the International Astronomical Union, 13, 175–178 Bassa, C., Pleunis, Z., & Hessels, J.
Vera: 2017, Astronomy and Computing, 18, 40 Braga, C.
Paper summary: Vera: The main thesis of The EDD Radio Astronomy Backend Framework is presenting a softwar" architecture designed to create reusable scalable real time radio astronomy backends on commodity hardware by separating common core services from observing mode plugins.
Jocelyn: It claims that this separation solves the problem where traditional backend instrumentation was too tightly coupled to specific observing modes, which limits reuse between observatories and science cases.
Subrahmanyan: So the central claim is about flexibility and reusability, allowing you to deploy new observing modes rapidly without needing to rebuild the entire instrument software for each case.
Vera: They describe this system as a modular distributed system consisting of loosely coupled processing and control elements, with persistent services like control and interfacing, and ephemeral pipelines that execute data processing or resource handling tasks.
Jocelyn: These pipelines are organized into observing modes by connecting them through well defined data streams forming a directed acyclic graph specified by a provision description.
Subrahmanyan: This DAG structure is key because it ensures deterministic configuration and execution across the system, which means the flow of data is predictable based on how those connections are made.
Vera: The entire operation at each site is coordinated by a Master Controller that manages the lifecycle of pipelines and services and enforces the system state model.
Jocelyn: All EDD pipelines adhere to a common state model, organized as a hierarchy of three control loops corresponding to distinct reconfiguration timescales in radio astronomical observations.
Subrahmanyan: This separation between parameters defined at pipeline initialization and those applied on a per observation or per scan basis allows for efficient reconfiguration without needing full reinitialization.
Vera: The framework also defines failure modes: an error state, a stalled state, and a panic state that indicate recovery requirements for the system.
Jocelyn: They mention that control communication relies on imperative command messages issued by the Master Controller to pipelines defining state transitions within this model.
Subrahmanyan: The paper sets up the context by noting that advances in COTS hardware and networking, like forty GbE and one hundred GbE interconnects, enable the use of commercial off-the-shelf hardware for high data rate low latency signal processing <ref:2609.11177#pg2,hardware for high data rate low latency signal processing>.
Conclusion: Vera: So looking at The EDD Radio Astronomy Backend Framework, it’s essentially a comprehensive way to build real time radio astronomy backends that are adaptable to different science needs without having to rewrite the main software for every telescope or instrument.
Jocelyn: It’s about creating a common foundation where you can plug in specific observing modes—like spectroscopy or pulsar timing—as just plugins onto this core system.
Subrahmanyan: The implication is that this significantly reduces the development and maintenance effort required when working across different observatories because you aren't starting from scratch every time.
Vera: It means that infrastructure can be shared more effectively, allowing multiple science cases to run commensally on the same underlying hardware.
Jocelyn: The paper shows how this framework successfully supports operations on both single dishes and larger interferometers while accommodating substantially different hardware and network setups.
Subrahmanyan: It’s a community resource because it provides a maintainable way for developing real time backends for broadband radio astronomy instruments, which is important for the long term.
Ewan D. Barr, Amit Bansod, Jan Behrend, Weiwei Chen, Niclas Esser, Yungpen Men, Tobias Winchen, Jason Wu, Uwe Bach, Claus Connot, Matthias Heininger, Jedrzej Jawor, Christoph Kasemann, Michael Kramer, Alex Kraus, Eddy Nussbaum, Oliver Polch, Jan Wagner
Max Planck Institute for Radio Astronomy · Huazhong University of Science and Technology · Jodrell Bank Centre for Astrophysics · The University of Manchester
astro-ph.IM
Submitted: 2026-09-10
Updated: 2026-09-10
Comments: Accepted for publication in Sect. 13. Astronomical instrumentation of Astronomy and Astrophysics (A&A)
Journal ref: Astronomy & Astrophysics, 714, A423 (2026)
DOI: 10.1051/0004-6361/202662135
Code: https://github.com/ska-sa/aiokatcp
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 94/100
The gist: The EDD Radio Astronomy Backend Framework presents a software-defined architecture designed to create reusable…
Key concepts
- Framework Design and Architecture
- The EDD backend is a distributed system made of loosely coupled processing services and ephemeral pipelines. Services handle persistent tasks like control, while pipelines execute specific data-processing or resource-handling jobs. These are organized into observing modes connected by data streams forming a directed acyclic graph (DAG).
- Operational State Model
- Pipelines follow a common state model that dictates their lifecycle, moving through stages like unprovisioned, idle, configured, and measuring. This model is structured around three control loops representing different reconfiguration timescales. The Master Controller issues explicit commands to trigger these transitions.
- Data Flow and Communication
- Communication is split into imperative control messages (from the Master Controller) and data streams for processing. Data exchange between pipelines primarily uses IP multicast over Ethernet for efficient fan-out. This design keeps the architecture agnostic about the underlying transport mechanism, allowing flexible deployment.
- Observational Capabilities and Deployments
- The framework supports diverse observing modes via plugins, including spectroscopy, pulsar timing, and VLBI. For example, spectroscopy uses GPU acceleration for simultaneous intensity and polarization recovery. It has been successfully deployed at four major sites ranging from single dishes to large interferometric arrays.
Terminology
Summary
The EDD Radio Astronomy Backend Framework presents a software-defined architecture designed to create reusable, scalable real-time radio astronomy backends on commodity hardware by separating common core services from observing mode plugins
Framework Design and Architecture
The EDD backend is structured as a modular, distributed system consisting of loosely coupled processing and control elements The system is composed of services, which are persistent components providing functionality like control, interfacing, and monitoring, and pipelines, which are ephemeral components executing data-processing or resource-handling tasks Pipelines are organized into observing modes by connecting them through well-defined data streams forming a directed acyclic graph (DAG) specified by a provision description The operation of the system at each site is coordinated by a Master Controller, which manages the lifecycle of pipelines and services and enforces the system state model
Operational State Model
All EDD pipelines adhere to a common state model that governs their lifecycle and operational control This model is organized as a hierarchy of three control loops corresponding to distinct reconfiguration timescales in radio astronomical observations The pipeline transitions between states such as unprovisioned, idle, configured, and measuring based on explicit commands from the Master Controller Specific commands like capture start transition pipelines into operational states depending on their class Failure modes include an error state, a stalled state, and a panic state indicating recovery requirements
Data Flow and Communication
Communication within the system is separated into control and data channels Control communication relies on imperative command messages issued by the Master Controller to pipelines defining state transitions Dataflow is realized through data streams connecting pipelines in the processing graph, where these streams define the interfaces and specify how data is exchanged The architecture is intentionally agnostic with respect to underlying data transport mechanisms, allowing pipelines to exchange data using various methods depending on deployment needs The primary method utilized is IP multicast over Ethernet for efficient fan-out and decoupling producers from consumers
Observational Capabilities and Deployments
The framework currently supports a broad range of observing modes through its plugin architecture, including spectroscopy, pulsar timing and searching, baseband recording, VLBI, correlation, and beamforming Representative modes include spectroscopy and spectropolarimetry which uses GPU-accelerated channelisation for simultaneous intensity and polarisation product recovery Pulsar timing supports two ingest paths depending on data rate, utilizing a CASPER-based FPGA polyphase filterbank plugin for channelisation in higher data rate scenarios The framework has been deployed at four sites: the Effelsberg 100-m radio telescope (EDGAR), SKAMPI, the Thai National Radio Telescope (TNRT), and the ARGOS interferometric prototype array
Performance and Comparative Results
When coupled with in-receiver digitisation, the EDD backend at Effelsberg provides ∼ 30% improvement in pulsar time-of-arrival estimation compared with previous instruments when using the same observing bandwidth and integration time For the P217mm receiver comparison, EDD showed that measured signal-to-noise ratios increase by 20–110% and corresponding pulse time-ofarrival uncertainties are reduced by approximately 20–40% compared to the legacy PSRIX backend This improvement is primarily attributed to digitising close to the receiver, which avoids signal degradation over the analogue transmission path The framework demonstrates scalability across different hardware architectures, including disaggregated, converged, and hyper-converged configurations at various deployments
Implementation and Maintenance
Deployment and provisioning of observation modes are managed using Ansible for configuration management and application deployment The EDD core and plugins are packaged as Ansible collections, enabling a configuration-as-code approach where the full system state is version-controlled Services run in isolated Docker containers, providing decoupled environments that integrate into diverse computing setups without imposing constraints on host distributions Monitoring and logging are handled through a combination of in-memory and time-series databases visualized via Grafana, with logs collected centrally from Docker output using Loki Dependency management is addressed through containerisation, fixed software suites for the core and plugins, and building against fixed Linux distributions to ensure compatibility across the stack
Conclusion
The EDD backend provides a reusable open-source framework for developing and operating real-time radio astronomy backends by separating common services from site-specific configuration and observing-mode plugins It successfully supports operations on both single dishes and interferometers while accommodating substantially different hardware, network, and control-system environments The framework serves as a community resource for developing maintainable real-time backends for broadband radio astronomy instruments
Pruning redundant reasoning tokens
The EDD backend addresses an ongoing challenge in modern radio astronomy: wide-band, directly digitised receivers increasingly require backend systems that can be reconfigured, maintained, and extended without rebuilding the instrument software for each observing mode or telescope
How the training signal is built
The EDD framework builds its capabilities through a core framework providing essential services like the Master Controller and interface services, while plugins supply processing pipelines for specific observing modes These pipelines are composed into observing modes by connecting them via data streams forming a DAG that ensures deterministic configuration and execution
Where the gains come from
The performance gains observed in pulsar timing estimations stem from digitising close to the receiver, which avoids signal degradation over the analogue transmission path The framework's success lies in its ability to support multiple science cases commensally on shared infrastructure by separating common services and observing-mode plugins
Deployment and Provisioning of Observation Modes
The EDD backend is deployed using Ansible, which automates software provisioning, configuration management, and application deployment via playbooks Installation involves pulling container images from a central registry and generating configuration files from an inventory to achieve reproducibility through a configuration-as-code approach Observation modes are provisioned by the user using commands sent to the Master Controller, which executes Ansible playbooks to launch the required pipeline and service containers
Control and Protocols
Communication between the Master Controller and pipelines uses the KAroo Telescope Communication Protocol (KATCP) for imperative command messages defining state transitions Infrastructure-level operations are managed by Ansible over SSH, while KATCP provides the runtime control layer through which the Master Controller orchestrates pipeline behavior during observations
Data Transport
The EDD framework is agnostic to data transport mechanisms, primarily utilizing IP multicast over Ethernet for efficient publish-subscribe data exchange between pipelines The SPEAD protocol encodes self-describing items into UDP packet streams, supporting complex multidimensional data types suitable for dynamic pipeline topologies High-performance transmission is achieved using tools like eddio, which leverages DPDK for low-level packet handling to achieve high line-rate performance
Monitoring and Logging
For operational monitoring, every pipeline exposes default metrics and pipeline-specific data via KATCP sensors collected by sidecar monitoring services Collected data is stored in an in-memory database for low-latency access to current state and a time-series database for high-resolution scalar metrics like packet loss rates Aggregated data from both systems is visualized using Grafana, while logs are collected from the Docker daemon and ingested into Loki for centralized querying
Observational Modes
The EDD backend supports various modes such as spectroscopy and spectropolarimetry, which employs GPU-accelerated channelisation and cross-polarisation coherence detection Pulsar timing and searching support unchannelised or channelised dual-polarisation voltage streams, with a CASPER-based FPGA plugin handling subband distribution for higher data rates VLBI capabilities involve a GPU-accelerated digital down-converter (DDC) pipeline that extracts arbitrary sidebands and packages them in standard VDIF format
Acknowledgements
Part of the work was based on observations with the 100-m telescope of the MPIfR at Effelsberg The SKAMPI prototype telescope is a facility of the Max Planck Society and was established with assistance from SARAO This research received funding from the MPG and the European Union’s Horizon Europe research and innovation programme under grant agreements No 101093934 (RADIOBLOCKS) and 101094354 (ARGOS-CDS)
References
Barr, E. D. 2017, Proceedings of the International Astronomical Union, 13, 175–178 Bassa, C., Pleunis, Z., & Hessels, J. 2017, Astronomy and Computing, 18, 40 Braga, C. p. 0 Historically, such backend
instrumentation has been tightly coupled to telescope observing modes, limiting reuse between observatories and science cases. We present the Effelsberg Direct Digitisation (EDD) backend framework, a software-defined architecture for constructing
real-time radio astronomy backends on commodity off-the-shelf computing infrastructure. 2019), which have reduced the effort required to develop packetised, high-throughput signal-processing systems. More recent
facility-scale instruments, including the Cobalt GPU correlator
for LOFAR (Broekema et al. More recent
facility-scale instruments, including the Cobalt GPU correlator
for LOFAR (Broekema et al. 2018), the MeerKAT correlatorbeamformer (Callanan 2020; van der Byl et al. main ©ESO 2026
September 11, 2026
The EDD Radio Astronomy Backend Framework
Ewan D. Barr1
, Amit Bansod1
, Jan Behrend1
, Weiwei Chen1
, Niclas Esser1
, Yungpen Men1, 2
, Tobias Winchen1⋆,
Jason Wu1
, Uwe Bach1
, Claus Connot1
, Matthias Heininger1
, Jedrzej Jawor1
, Christoph Kasemann1
, Michael Kramer1, 3
,
Alex Kraus1
, Eddy Nussbaum1
, Oliver Polch1
, Jan Wagner1
, and Gundolf Wieching1
1 Max Planck Institute for Radio Astronomy, Auf dem Hügel 69, 53121 Bonn, Germany
2 Huazhong University of Science and Technology, 1037 Luoyu Road, Wuhan 430074, China
3
Jodrell Bank Centre for Astrophysics, The University of Manchester, Manchester M13 9PL, UK
Received XXX; accepted YYY
ABSTRACT
Context. However, in practice, the active processing capabilities within the EDD ecosystem have converged on
a common set of protocols and tooling, which we describe here. The primary data transport method is IP multicast (Deering 1989)
over Ethernet. 2025; Pearlman et al. 2025; Limaye et al. 2022) and the
Commensal Realtime ASKAP Fast Transient Coherent upgrade (CRACO; Wang et al. 2025), further illustrate the increasing use of hybrid FPGA/GPU architectures and high-speed EtherArticle number, page 1
arXiv:2609. main ©ESO 2026
September 11, 2026
The EDD Radio Astronomy Backend Framework
Ewan D. Barr1
, Amit Bansod1
, Jan Behrend1
, Weiwei Chen1
, Niclas Esser1
, Yungpen Men1, 2
, Tobias Winchen1⋆,
Jason Wu1
, Uwe Bach1
, Claus Connot1
, Matthias Heininger1
, Jedrzej Jawor1
, Christoph Kasemann1
, Michael Kramer1, 3
,
Alex Kraus1
, Eddy Nussbaum1
, Oliver Polch1
, Jan Wagner1
, and Gundolf Wieching1
1 Max Planck Institute for Radio Astronomy, Auf dem Hügel 69, 53121 Bonn, Germany
2 Huazhong University of Science and Technology, 1037 Luoyu Road, Wuhan 430074, China
3
Jodrell Bank Centre for Astrophysics, The University of Manchester, Manchester M13 9PL, UK
Received XXX; accepted YYY
ABSTRACT
Context. 2015) and the
GPU-based Medusa backend (Hobbs et al. 2020), have progressively adopted programmable digital processing architectures to
increase bandwidth, flexibility, and observing efficiency. main ©ESO 2026
September 11, 2026
The EDD Radio Astronomy Backend Framework
Ewan D. Barr1
, Amit Bansod1
, Jan Behrend1
, Weiwei Chen1
, Niclas Esser1
, Yungpen Men1, 2
, Tobias Winchen1⋆,
Jason Wu1
, Uwe Bach1
, Claus Connot1
, Matthias Heininger1
, Jedrzej Jawor1
, Christoph Kasemann1
, Michael Kramer1, 3
,
Alex Kraus1
, Eddy Nussbaum1
, Oliver Polch1
, Jan Wagner1
, and Gundolf Wieching1
1 Max Planck Institute for Radio Astronomy, Auf dem Hügel 69, 53121 Bonn, Germany
2 Huazhong University of Science and Technology, 1037 Luoyu Road, Wuhan 430074, China
3
Jodrell Bank Centre for Astrophysics, The University of Manchester, Manchester M13 9PL, UK
Received XXX; accepted YYY
ABSTRACT
Context. main ©ESO 2026
September 11, 2026
The EDD Radio Astronomy Backend Framework
Ewan D. Barr1
, Amit Bansod1
, Jan Behrend1
, Weiwei Chen1
, Niclas Esser1
, Yungpen Men1, 2
, Tobias Winchen1⋆,
Jason Wu1
, Uwe Bach1
, Claus Connot1
, Matthias Heininger1
, Jedrzej Jawor1
, Christoph Kasemann1
, Michael Kramer1, 3
,
Alex Kraus1
, Eddy Nussbaum1
, Oliver Polch1
, Jan Wagner1
, and Gundolf Wieching1
1 Max Planck Institute for Radio Astronomy, Auf dem Hügel 69, 53121 Bonn, Germany
2 Huazhong University of Science and Technology, 1037 Luoyu Road, Wuhan 430074, China
3
Jodrell Bank Centre for Astrophysics, The University of Manchester, Manchester M13 9PL, UK
Received XXX; accepted YYY
ABSTRACT
Context. Observing Modes
The EDD backend currently supports a broad range of observing
modes through its plugin architecture. Available capabilities include digitisation and packetisation for several digital receiver systems, channelisation, spectroscopy and spectropolarimetry, pulsar
timing, pulsar and transient searching, baseband recording, Very
Long Baseline Interferometry (VLBI), correlation and beamforming, and various output formatting pipelines for formats including
HDF521, PSRFITS (Hotan et al. Both ingest paths produce data in a common
internal format and are handled transparently by the downstream
processing stages. In timing mode, the pipeline performs GPU-accelerated
coherent dedispersion and phase-aligned folding using
dspsr (van Straten & Bailes 2011) and tempo2 (Hobbs et al. de
The evolution of such digital backends has historically been
driven by the needs of specific experiments and observing modes,
with early systems such as autocorrelators and digital filterbanks
typically implemented on custom hardware. Subsequent generations of instruments, including spectrometers such as the
XFFTS (Klein et al. Subsequent generations of instruments, including spectrometers such as the
XFFTS (Klein et al. 2012), VLBI backends such as the DBBC
family (Tuccari et al. 2025; Pearlman et al. 2025; Limaye et al. Astronomy & Astrophysics manuscript no. main ©ESO 2026
September 11, 2026
The EDD Radio Astronomy Backend Framework
Ewan D. Observations using this mode have
contributed to studies of scintillation (Main et al. 2023a,b), scattering screen dynamics (Sprenger et al. The ARGOS prototype backend differs from the other EDD
deployments in several respects. At the frontend, ARGOS employs an RFSoC-based digitiser and F-engine (Men et al. From these two sources, which
are typically YAML4 files, Ansible idempotently executes all
required actions on the target machines so that the complete
installation of an EDD instance is reproducible. This realises a
configuration-as-code approach (Morris 2025) in which the full
system state is version-controlled, providing both reproducibility
and a record of the deployment history for data provenance. From these two sources, which
are typically YAML4 files, Ansible idempotently executes all
required actions on the target machines so that the complete
installation of an EDD instance is reproducible. This realises a
configuration-as-code approach (Morris 2025) in which the full
system state is version-controlled, providing both reproducibility
and a record of the deployment history for data provenance. Spectrometer output data can be recorded directly to disk
in raw binary format or forwarded to dedicated downstream
pipelines. Separate plugins provide recording in HDF5 format and forwarding to an external APEX-compatible FITS
writer (Muders et al. Separate plugins provide recording in HDF5 format and forwarding to an external APEX-compatible FITS
writer (Muders et al. 2018), the latter enabling integration with
existing observatory data flows at Effelsberg. This publish-subscribe
model decouples producers from consumers and simplifies the
construction of complex processing graphs in which a single data
stream may feed multiple independent downstream stages. The
EDD core provides tooling for working with multicast traffic,
including automatic management of multicast group address allocation to avoid conflicts within a deployment. Unlike stateless REST-over-HTTP interfaces, KATCP maintains a persistent connection over which
devices can push sensor updates without being polled, avoiding the need for auxiliary feedback channels. The choice of
KATCP was motivated by prior experience with the protocol during development of the APSUSE and FBFUSE instruments for
MeerKAT (Barr 2017; Padmanabh et al. The choice of
KATCP was motivated by prior experience with the protocol during development of the APSUSE and FBFUSE instruments for
MeerKAT (Barr 2017; Padmanabh et al. 2023) and the availability of mature third-party libraries, in particular aiokatcp7
, which
provides a fully asynchronous Python implementation suitable
for integration into the EDD’s concurrent control architecture. 2018), broadband interferometric backends
such as CABB (Wilson et al. 2011), and general-purpose or multimode systems such as VEGAS (Prestage et al. 2011), and general-purpose or multimode systems such as VEGAS (Prestage et al. 2015) and the
GPU-based Medusa backend (Hobbs et al. At Effelsberg, all receiver configurations include a spectrometer instance feeding the telescope’s FITS
writer, providing a uniform spectroscopic capability across the observatory’s receiver suite. At TNRT, the spectrometer was the first
commissioned observing mode and has been used to detect OH
maser emission from a comet (Sakai et al. 2016),
and hardware platforms, e. UniBoard2 (Schoonderbeek et al. Observations using this mode have
contributed to studies of scintillation (Main et al. 2023a,b), scattering screen dynamics (Sprenger et al. Astronomy & Astrophysics manuscript no. main ©ESO 2026
September 11, 2026
The EDD Radio Astronomy Backend Framework
Ewan D. Subsequent generations of instruments, including spectrometers such as the
XFFTS (Klein et al. 2012), VLBI backends such as the DBBC
family (Tuccari et al. More recent
facility-scale instruments, including the Cobalt GPU correlator
for LOFAR (Broekema et al. 2018), the MeerKAT correlatorbeamformer (Callanan 2020; van der Byl et al. 2018), the MeerKAT correlatorbeamformer (Callanan 2020; van der Byl et al. 2022) and the
Commensal Realtime ASKAP Fast Transient Coherent upgrade (CRACO; Wang et al. Both ingest paths produce data in a common
internal format and are handled transparently by the downstream
processing stages. In timing mode, the pipeline performs GPU-accelerated
coherent dedispersion and phase-aligned folding using
dspsr (van Straten & Bailes 2011) and tempo2 (Hobbs et al. 2018), broadband interferometric backends
such as CABB (Wilson et al. 2011), and general-purpose or multimode systems such as VEGAS (Prestage et al. The VLBI plugin is deployed at both Effelsberg and
SKAMPI. At SKAMPI, it has been used for successful VLBI observations with international networks (Wongphechauxsorn et al. Astronomy & Astrophysics manuscript no. main ©ESO 2026
September 11, 2026
The EDD Radio Astronomy Backend Framework
Ewan D. 2025; Limaye et al. 2026), and
magnetar emission properties (Younes et al.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Backend Flexibility for Heterogeneous Data Streams: The improved AI system can support multiple science cases commensally on shared infrastructure by separating
common services, observing-mode plugins, and site-specific configuration.
This allows backend capabilities to be deployed across heterogeneous telescope environments without rebuilding instrument software for each mode. -
Real-time System Reconfiguration: The system can handle rapid changes in observation requirements through a hierarchy of three control loops governing pipeline state transitions. This separation between parameters defined at initialization and those applied on a per-observation or per-scan basis allows for
efficient reconfiguration of pipelines without requiring full reinitialization.
-
Hardware Agnostic Pipeline Execution: The architecture is intentionally agnostic with respect to the underlying data transport mechanism, allowing pipelines to exchange data using
a variety of methods depending on the requirements of the deployment environment and the processing task.
This enables portability across different telescope environments by decoupling processing from specific hardware constraints. -
Automated Infrastructure Provisioning: The system can achieve reproducible and consistent deployments by using Ansible to encode deployment steps as playbooks, realizing a
configuration-as-code approach
wherethe full system state is version-controlled.
This ensures that resource assignments and service descriptions are maintained consistently between installation and runtime provisioning. -
High-Throughput Data Ingest: The system can handle high data rates by supporting two complementary ingest paths for pulsar observations, distributing the signal across multiple processing nodes using
multicast data transport.
This enables processing of higher data rates exceeding single-node capacity through an upstream channelisation stage that divides the signal into frequency subbands. -
Enhanced Timing Precision: When coupled with in-receiver digitisation, the system can provide
significant improvements in sensitivity and timing precision by avoiding degradation in the analogue transmission path.
This avoids limitations imposed by analogue signal transport over long paths, leading to a30% improvement in pulsar time-of-arrival estimation compared with previous instruments.
-
Integrated Monitoring and Diagnostics: The system provides a unified view of operational health through sidecar monitoring services that inject data into an in-memory database for low-latency access and a time-series database for high-resolution metrics. This allows human operators to
provide a realtime view of the system for human and automated operators
by visualizing aggregated data using Grafana.
Abstract
Modern digital radio astronomy receivers produce increasingly wide-bandwidth, high bit-rate data streams that necessitate the development of flexible, scalable, and maintainable backend processing and recording systems. Historically, such backend instrumentation has been tightly coupled to telescope observing modes, limiting reuse between observatories and science cases. We present the Effelsberg Direct Digitisation (EDD) backend framework, a software-defined architecture for constructing real-time radio astronomy backends on commodity off-the-shelf computing infrastructure. We describe its design, implementation, supported observing modes, and operational deployments. EDD separates a common core framework from plugin-provided observing capabilities. The core provides orchestration, telescope interfaces, pipeline lifecycle management, monitoring, and deployment tooling, while plugins implement processing pipelines for specific observing modes. The framework is designed to support both single-dish and interferometric instruments through site-specific configuration and plugin selection. EDD currently supports spectroscopy and spectropolarimetry, pulsar timing and searching, baseband recording, very long baseline interferometry, correlation, and beamforming. Operational deployments include the Effelsberg 100-m telescope, the SKA-MPI prototype dish, the Thai National Radio Telescope, and the ARGOS interferometric prototype array. By separating common services, observing-mode plugins, and site-specific configuration, it allows backend capabilities to be deployed across heterogeneous telescope environments and provides a community resource for broadband radio astronomy instrumentation.
Sources
Related papers
- On-sky demonstration of a vector Zernike wavefront sensor in a cascaded adaptive optics system
- Euclid Data Release 1 (DR1): Calibration strategy and observations
- The General Antiparticle Spectrometer (GAPS) Antarctic Balloon Payload
- Detection and excision of radio-frequency interference in radio arrays
- Unified matched-filter and excess-power pipeline for gravitational-wave detector characterization
- Ushering in sub-arcsecond angular resolution observatories in the far-infrared: The case for a double-Fourier interferometer