Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on CPUs and GPUs
summary
The gist
Batched matched filtering and cross-correlation over large template banks are a primary computational cost in gravitational-wave astronomy, observational seismology…
In short
The authors developed a hierarchical screening method combined with operator fusion for batched matched filtering. This approach significantly reduces memory traffic and computation by fusing operations and using short, low-frequency transforms before full bandwidth evaluation. The method achieves massive speedups across CPUs and GPUs while maintaining accuracy by controlling the false-dismissal rate.
Key concepts
- Conventional Pipeline Overhead
- The standard method involves reading intermediate results like the product of data and template ($ ext{P} = ext{Dd} ullet ext{H}^*t$) into memory buffers, writing them back, and then re-reading them. This constant swapping between memory and processing units creates significant memory traffic—48N bytes per pair for $N=4096$.
- Operator Fusion
- This technique combines operations, such as the conjugate multiplication into the first inverse transform stage and the peak search into the last stage. By keeping intermediate results in fast on-chip memory (registers or cache), it eliminates many memory reads and writes, reducing traffic from 48N bytes to about 65.5 KB per pair.
- Hierarchical Screening
- Instead of processing everything at once, this method uses a sequence of short frequency transforms. These stages are tuned based on the expected noise profile to decide which pairs need full evaluation, effectively filtering out many noise-only pairs early in the process.
Terminology used across episodes
This episode discusses
- Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on CPUs and GPUs · Paper Radio
- Fast Training of Convolutional Networks through FFTs
- Fast Convolutional Nets With fbfft: A GPU Performance Evaluation
- FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
- Sub-Nyquist Radar: Principles and Prototypes
- Pulscan: Binary pulsar detection using unmatched filters on NVIDIA GPUs
The paper
Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on CPUs and GPUs · Read on arXiv
Alexander H. Nitza, Keisi Kacanjaa, Collin D. Capano
Department of Physics, Syracuse University
Transcript
Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.
Vera: We're looking at the paper: "Peak Performance in Matched Filtering".
Jocelyn: Here's the gist of the paper. Batched matched filtering and cross-correlation over large template banks are a primary computational cost in gravitational-wave astronomy, observational seismology, and radio transient searches The authors develop a hierarchical screening method that takes advantage of this using a sequence of short low-frequency transforms before evaluating the full bandwidth This approach reduces memory traffic and transform work by fusing the conjugate product into the first stage of the inverse transform and the peak search into the last, while tiling each batch so that a spectrum loaded into cache serves many data–template pairs Conventional Pipeline Overhead The conventional pipeline involves reading and writing intermediate arrays between separate pipeline kernels, which results in significant memory traffic Specifically, a conventional pipeline writes the product P = Dd ⊙ H∗t and the full correlation series z
τ: to memory buffers (cache or DRAM) and reads both back, transferring 48N bytes per pair This leads to an arithmetic intensity of Iunfused ≈ 1.38 FLOP/byte for N = 4096 Operator Fusion and Tiling Benefits The authors fuse the conjugate multiplication into the first butterfly stage of the inverse transform and the binned peak search maxτ z
τ: 2 into the final butterfly stage, keeping intermediate states in registers, L1 cache, or GPU local data share This fusion reduces memory traffic per pair from 48N bytes to 65.5 KB for N = 4096 Furthermore, in a K × K cache tile, the K data spectra and K template spectra (2K spectra, 16KN bytes) are loaded once and serve all K2 pairs in the tile This reduces memory traffic outside the cache tile by R ≈ 3K× (5) relative to the 48N bytes/pair of an unfused pipeline Statistically Controlled Screening The authors build multi-stage frequency-band screening directly into the matched-filter primitive itself and automatically determine optimal stage bandwidths and thresholds from the expected spectral power profile and matched-filter noise autocovariance A ∝ F −1 H2 / S n to satisfy a statistically defined target false-dismissal rate α The conditional false-dismissal rate PFD(ρc) is calculated by considering the probability that a signal reaching ρth in the full filter is rejected by any screening tier They demonstrated that for a binary-neutron-star template whitened by an Advanced LIGO noise curve at N = 4096 and ρth = 5.5, the first 512 bins (f ≤ 256 Hz) already capture 96.5% of the full-band SNR Performance Benchmarks The hierarchical screening method yields significant speedups across different hardware, demonstrating a 26× speedup over AOCL-FFTW on the CPU, 21× over rocFFT on the Radeon 8060S GPU, and 19× over MLX FFT on the Apple M2 GPU On a single-core CPU, hierarchical screening reaches break-even with fused peak-only filtering near ρth = 4 (1.0–1.3×), and rises smoothly with detection threshold to 6–18× at ρth = 5.5 GPU performance shows that fused peak-only filtering is 6–13× faster than the complete unfused HIP + rocFFT pipeline Combining operator fusion with covariance-calibrated screening reduces both memory traffic and transform work, accelerating batched matched filtering on CPUs and GPUs without sacrificing single-precision accuracy or signal recovery beyond a user-specified false-dismissal rate The authors suggest that coarse screening stages may operate in reduced-precision formats (float16, bfloat16, or fp8) to leverage the tiled matrix and tensor units of modern GPUs and neural processors This approach offers the potential to run bank-wide analyses on commodity workstations and consumer GPUs Page 4 --- (a) Conventional pipeline Memory buffers (cache / DRAM) Dd, Ht (in memory) Product P=Dd ⊙H∗t Inverse FFT Peak search (ρ∗, τ∗) read 16N write 8N read 8N write 8N read 8N 48N B/pair (196.6 KB at N=4096) Page 5 --- (b) K×K tile Data d Templates t2K cached spectra →K2 correlations →K2 peak records Page 7 --- Figure 1: Memory traffic in unfused, fused, and tiled matched filtering.
Vera: (a) A conventional pipeline writes the product P = Dd ⊙ H∗t and the full correlation series z
τ: to memory buffers (cache or DRAM) and reads both back, transferring 48N bytes per pair.
Jocelyn: The fused filter applies the conjugate product in the first butterfly and the z2 maximum in the last, keeping intermediate states in registers, L1 cache, or GPU local memory so only the 16N bytes of input spectra are read (b) In a K × K cache tile, 2K cached input spectra serve K2 correlations →K2 peak records Page 8 --- (c) Traffic per pair (N=4096) Fused + tiled 2.0 KB Conventional 65.5 KB Page 9 --- (d) Scaling with transform length N AOCL-FFTW 5.3 (inverse FFT) Fused (full output) Fused (peak only) Hierarchical (ρth =5.5) Evaluation mode (N=4096) 0 1 2 3 4 5 6 78–26× Page 10 --- Figure 3: Single-core CPU performance on the AMD Ryzen AI Max+ 395 (Zen5, AVX-512).
Vera: (a) Time per pair across transform lengths N ∈
1024, 65536: for AMD AOCL-FFTW 5.3 (FFTW PATIENT) and the fused full-output and peak-only filters; shaded bands span the 10th–90th percentile of timing blocks (b) Throughput in millions of pairs per second at N = 4096 and ρth = 5.5 in Gaussian noise with the PyCBC binary-neutron-star reference profile, showing AOCL-FFTW, fused full-output, fused peak-only, and hierarchical screening with false-dismissal budgets α ∈
10−4, 10−2: overlaid (speedup over AOCL-FFTW in parentheses) Page 4 --- (c) False-dismissal validation (ρth =5.5) Matched reference Heterogeneous f γ bank α=10−2 α=10−3 α=10−4 40 128 512 2048 Frequency bin k Page 6 --- (c) Measured conditional FDR with Wilson 95% confidence intervals against targets α ∈
10−2, 10−4: at ρth =5.5: left of the divider shows synthetic templates matched to their own reference envelopes across inspiral, f −1, f0, and f+1 spectra; right of the divider shows a heterogeneous power-law stress bank (fγ, γ ∈
−2.33, −1.33: ) comparing a single edge-template reference, a single bank-mean reference, and four reference groups partitioned by spectral slope Page 8 --- Table 1: Single-precision fused peakonly filter against a double-precision (float64) reference across 640 trials per transform length spanning injected SNRs from 0 (noise only) to 100.
Title and authors: Vera: So this paper, "Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on CPUs and GPUs," it’s talking about how we can make batched matched filtering much faster when we have really big template banks, which is a huge cost right now in gravitational-wave astronomy and radio transient searches.
Jocelyn: Yeah, it’s about tackling that computational cost by using this hierarchical screening method that uses short low frequency transforms before looking at the full bandwidth of the signal. It sounds like they’re trying to cut down on memory traffic and how much transform work we do by fusing things together in a specific way.
Subrahmanyan: From a theoretical side, if you can speed up these correlation calculations over large template banks, it means we can test more complex astrophysical models against the data, which is what I care about most regarding structure formation and cosmology.
Vera: The paper breaks down the conventional pipeline overhead by showing how much memory traffic there is when you’re just reading and writing intermediate arrays between different pipeline kernels.
Jocelyn: It says that a conventional pipeline writes the product P equals Dd times H star t, and then the full correlation series z of tau to memory buffers for every template pair, which means it transfers forty eight N bytes per pair at N equals four thousand ninety-six.
Subrahmanyan: Forty eight N bytes per pair is a lot when you're dealing with massive banks of templates; that really highlights the need for optimization in any large-scale simulation or search.
Vera: They then show how they fuse the conjugate product into the first part of the inverse transform and put the peak search—that maximum tau squared—into the last part, which keeps those intermediate states in registers, L1 cache, or GPU local memory instead of moving them all to DRAM.
Jocelyn: And that fusion drops that traffic per pair down from forty eight N bytes to sixty five point five kilobytes for N equals four thousand ninety-six which is a big reduction in memory movement.
Subrahmanyan: Shifting the bottleneck from DRAM bandwidth to on-chip cache bandwidth is a significant engineering win because it shows the calculation can stay closer to the processing unit.
Vera: They also introduce two dimensional cache tiling where they load twenty K cached input spectra and templates and reuse those twenty K spectra for all of the four thousand ninety-six correlation records within that tile.
Jocelyn: That tiling mechanism means they reduce memory traffic outside the cache tile by a factor of roughly three times compared to the forty eight N bytes per pair you saw earlier, because they only load those input spectra once.
Subrahmanyan: So, it’s this way they handle larger template banks efficiently by reusing the input data already loaded into fast memory rather than reloading it constantly.
Title and authors: Vera: Then there is this statistically controlled screening part where they build multi-stage frequency band screening directly into the matched filter primitive itself, automatically picking the best stage bandwidths and thresholds based on the expected spectral power profile and noise autocovariance.
Jocelyn: They use a target false dismissal rate alpha to define how much we want to reject signal candidates at each tier, and they calculated that for a binary neutron star template whitened by Advanced LIGO noise at N equals four thousand ninety-six the first five hundred twelve bins, which is up to two hundred fifty hertz, already capture ninety six point five percent of the full band signal-to-noise ratio.
Subrahmanyan: That result suggests that you don't need to process the entire bandwidth immediately; you can filter out most of the noise early based on statistical expectations derived from the noise curve itself.
Vera: The paper shows performance benchmarks across different hardware, demonstrating a speedup of twenty six times over AOCL FFTW on a CPU, and nineteen times over MLX FFT on an Apple M2 GPU for single core performance.
Jocelyn: It also shows that fused peak only filtering is six to thirteen times faster than the complete unfused HIP plus rocFFT pipeline when looking at GPU performance results.
Subrahmanyan: The scaling with transform length N is impressive, showing a speedup of twenty six times over AOCL FFTW on the CPU and nineteen times over MLX FFT on the Apple M2 GPU for N equals four thousand ninety-six.
Vera: The paper also provides false dismissal validation at fifty point five threshold, where they show that they can control the conditional false dismissal rate with Wilson ninety-five percent confidence intervals against targets alpha equal to ten to the negative two, three to the negative three, and four times ten to the negative four across different frequency bins.
Jocelyn: That validation shows that you can reliably hit your desired false dismissal rate even when you are dealing with a heterogeneous power law stress bank, comparing a single edge template reference against a bank mean reference or four different groups.
Subrahmanyan: It’s interesting that they show the conditional false dismissal rate calculation involves considering the probability that a signal reaching a certain threshold is rejected by any screening tier, which connects the statistical filtering directly to the physics of how we expect signals to look.
Vera: Overall, this work shows how combining operator fusion with covariance calibrated screening reduces both memory traffic and transform work without sacrificing single precision accuracy or signal recovery beyond what you set in your false dismissal rate.
Jocelyn: They also suggest that coarse screening stages might operate in reduced precision formats like float sixteen or bfloat sixteen to take advantage of the tiled matrix and tensor units on modern GPUs and neural processors.
Title and authors: Subrahmanyan: That idea of running coarse screening stages in lower precision formats could be really useful for running bank wide analyses on commodity workstations and consumer GPUs, which opens up access to a lot more data processing power.
Vera: So, to wrap up, this paper "Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on CPUs and GPUs" shows that by fusing the conjugate product into the first inverse transform butterflies and putting the peak reduction in the last butterfly, plus using 2D cache tiling to serve multiple data template pairs from cached spectra, we can move this process from being memory bandwidth bound to being cache bandwidth bound <ref:2611.17229#pg1,Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on>.
Jocelyn: And on top of that, they integrate a hierarchical screening method that determines optimal stage bandwidths based on the spectral power profile and noise autocovariance to control the false dismissal rate statistically.
Subrahmanyan: This combination means we are getting significant speedups, like twenty six times over AOCL FFTW on a CPU, while still maintaining single precision accuracy for signal recovery up to a user specified false dismissal rate.
Vera: It also shows that fusing peak only filtering can be six to thirteen times faster than the full unfused pipeline when looking at GPU performance benchmarks.
Jocelyn: The implication is that you can achieve this acceleration without needing more expensive hardware, just by being smarter about how you handle the data flow and what parts of the search are most important at different stages.
Subrahmanyan: What this means for me is that we can run much bigger template banks, which will let us test more complex structure formation models against the observational constraints we get from things like binary neutron stars.
Vera: So, to conclude on this paper about "Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on CPUs and GPUs," it’s a method that combines operator fusion with covariance calibrated screening to reduce memory traffic and transform work for batched matched filtering.
Jocelyn: It’s about using 2D cache tiling where twenty K cached input spectra serve four thousand ninety-six correlations, which is a big win for data reuse <ref:2611.17229#pg1>.
Subrahmanyan: It's about scaling the search effectively across different hardware, showing speedups of twenty six times over AOCL FFTW on a CPU and nineteen times over MLX FFT on the Apple M2 GPU for N equals four thousand ninety-six.
Vera: And they also show that this approach can be scaled up by using reduced precision formats in coarse screening stages, which could potentially run bank wide analyses on commodity workstations.
Jocelyn: It’s a method that balances computational cost with statistical control over false dismissal rates, which is exactly what we need for reliable searches.
The paper's summary: Vera: So we're talking about this paper that tackles how to make those large template bank searches on CPUs and GPUs way faster by fusing the correlation steps together and using statistical screening to cut down on work early in the process.
Jocelyn: Right, it takes something that used to be a huge memory bottleneck, moving data back and forth between different pipeline parts, and tries to fix that by keeping stuff in registers or local cache instead of constantly reading from DRAM.
Subrahmanyan: From a theoretical standpoint, it’s about scaling the search efficiently for things like gravitational waves or transients where you have thousands of templates to check against new data. If you can make that faster, we can test more complex models.
Vera: The paper shows they took this conventional pipeline and reduced the memory traffic per pair from forty eight N bytes down to sixty five point five kilobytes when the transform length N was four thousand ninety-six.
Jocelyn: That's a big jump in efficiency because it moves the problem away from being about how fast you can move data around to how much computation you can do with what's already in fast memory.
Subrahmanyan: And they’re not just doing that; they’re using this hierarchical screening method where they decide which frequency bands to look at based on the noise profile and the expected signal shape, targeting a specific false dismissal rate alpha.
Vera: So, instead of running the full bandwidth search on every single pair right away, you run short transforms first to filter out most of the noise before committing to a full search.
Jocelyn: That statistical control is important because it means you can keep the accuracy up while still cutting down on the work that’s just noise.
Subrahmanyan: And they showed that for a binary neutron star template at five point five threshold, just looking at the first two hundred fifty hertz captures ninety six point five percent of the signal we expect.
Vera: That means you don't need to process everything if you know where the signal is most likely to be.
Jocelyn: The hardware results are pretty telling too; they show speedups like twenty six times over AOCL FFTW on a CPU and nineteen times over MLX FFT on the Apple M2 GPU for single core performance.
Subrahmanyan: That scaling across different processors suggests this method is really robust, not tied to one specific machine.
Vera: The paper also shows that fused peak only filtering can be six to thirteen times faster than the full unfused pipeline when you look at GPU performance results.
Jocelyn: So even without the full correlation series, just finding the peaks is a significant speed boost for high throughput searches.
Subrahmanyan: What this implies is that we can push batched matched filtering much further into real-time or near-real-time analysis across wider template banks than we could before.
Vera: The authors also suggest you can use reduced precision formats like float sixteen or bfloat sixteen for those coarse screening stages to take advantage of modern GPU tensor units.
Jocelyn: That idea of using lower precision for the initial filtering stages is interesting because it could really help run these large analyses on more common hardware, not just supercomputers.
Subrahmanyan: It’s a path toward making this kind of high-throughput analysis accessible to a wider range of research groups.
Vera: So, in short, they combined operator fusion for memory efficiency with statistical screening to control false dismissal rates and achieve substantial speedups across different hardware platforms.
Jocelyn: And the next thing we need to look at is how this method actually handles the complexity of choosing the best screening sequence based on processor cost and signal characteristics.
The paper's improvements: Vera: So we're talking about what they suggest to make this system even better than just the baseline fused approach and hierarchical screening they first showed us today.
Jocelyn: It’s about taking those initial speedups and figuring out how to make the whole search chain more adaptive, so it chooses its own path based on what it sees.
Subrahmanyan: Adaptive screening is key for cosmology; if we can dynamically adjust our search strategy based on the noise environment or signal characteristics, we get a better chance of finding rare events or subtle structure.
Vera: They suggest building a model to minimize the cost by evaluating different screening chains, which they call minimizing C equals c1 plus X k i equals two ci pi i minus one plus cfine pi k.
Jocelyn: That sounds like a way to dynamically select the fastest path through all those screening stages based on what's happening in the data right now.
Subrahmanyan: It’s about finding the optimal sequence based on expected cost against the modeled cumulative noise survival fractions pi i, which connects directly to how much work we save for every potential detection.
Vera: And they also look at using reduced precision formats, like float sixteen or bfloat sixteen, specifically for those initial coarse screening stages.
Jocelyn: That’s really smart because if you can run the preliminary filtering stages in lower precision, it could potentially halve the memory footprint for those low frequency prefix spectra.
Subrahmanyan: If that works well, it opens up running these large template bank analyses on commodity workstations and consumer GPUs, which is a huge thing for accessibility.
Vera: The caveat they mention is that this reduced precision only applies to the coarse screening stages; the final full-bandwidth filter still needs to be in single precision float32 for accurate signal recovery.
Jocelyn: So you use lower precision to decide if something is worth looking at, but then you run the high precision search when it matters most.
Subrahmanyan: That separation makes sense; we prioritize speed where it helps us discard noise, and accuracy where we need to confirm a real signal.
Vera: What this means for the sky is that these techniques allow us to scale up our template banks so much that we can test more complex astrophysical models against the observational constraints we get from things like binary neutron stars.
Jocelyn: It really shows how you can balance the need for high throughput with maintaining a strict control over false dismissal rates across a massive search space.
Subrahmanyan: It’s about making the search process smarter by dynamically choosing its screening strategy, which helps us find those subtle signals that might otherwise be lost in the noise.
Conclusion: Vera: So we’re wrapping up this session on "Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on CPUs and GPUs."
Jocelyn: It’s about how combining operator fusion with covariance calibrated screening helps cut down memory traffic and transform work for batched matched filtering.
Subrahmanyan: For cosmology, this means we can test more complex models against the observational constraints we get from gravitational waves or radio transients.
Vera: The key thing is that they reduced the memory traffic from forty eight N bytes per pair down to sixty five point five kilobytes when N was four thousand ninety-six.
Jocelyn: That’s a big jump because it moves the bottleneck away from DRAM bandwidth and onto on-chip cache bandwidth.
Subrahmanyan: Scaling up our template banks so much means we can test more complex structure formation models against the observational constraints we get from things like binary neutron stars.
Vera: And they showed that fusing peak only filtering can be six to thirteen times faster than the full unfused pipeline when you look at GPU performance results.
Jocelyn: It’s about achieving high throughput without needing more expensive hardware for every single calculation.
Subrahmanyan: This method is really robust, not tied to one specific machine, because it scales across different processors like CPUs and GPUs.
Vera: They also showed that you can use reduced precision formats for the initial coarse screening stages to take advantage of modern GPU tensor units.
Jocelyn: It’s about making this kind of high throughput analysis accessible on more common hardware, not just supercomputers.
Subrahmanyan: If we can make this process smarter by dynamically choosing its screening strategy, it helps us find those subtle signals that might otherwise be lost in the noise.
Vera: So, in short, they combined operator fusion with covariance calibrated screening to reduce memory traffic and transform work for batched matched filtering.
Jocelyn: It’s about using 2D cache tiling where twenty K cached input spectra serve four thousand ninety-six correlations <ref:2611.17229#pg1>.
Subrahmanyan: It's about scaling the search effectively across different hardware, showing speedups of twenty six times over AOCL FFTW on a CPU and nineteen times over MLX FFT on the Apple M2 GPU for N equals four thousand ninety-six.
Vera: And they also show that this approach can be scaled up by using reduced precision formats in coarse screening stages.
Jocelyn: It’s a method that balances computational cost with statistical control over false dismissal rates, which is exactly what we need for reliable searches.
Subrahmanyan: This work on "Peak Performance in Matched Filtering: Fused Correlation and Statistically Controlled Screening on CPUs and GPUs" points toward a future where we can run much bigger template banks without the current memory limitations.
Vera: It's a solid piece of engineering that lets us push the limits of what we can do with our current hardware for these huge searches.
Jocelyn: Next time, we’ll look at how this speedup translates into actual sensitivity improvements for finding those faint signals.
More episodes
- 2607.17929-gevolution 2.0: GPU-accelerated relativistic N-body simulations for cosmology
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2509.02657-On the synergetic use of Ariel and JWST for exoplanet atmospheric science
- 2609.10042-Modelling Fourier filtering wavefront sensors for PSD-based methods: the AOPERA tool
- 2511.05639-Performance of the AstroPix Prototype Module for the Barrel Imaging Calorimeter at the ePIC Detector and in Space-Based Payloads
- 2610.19909-RPCA: a Resolved PSF reconstruction method for undersampled images
- 2512.03440-An Analysis of LIGO Glitches Using t-SNE During the First Part of the Fourth LIGO-Virgo-KAGRA Observing Run
- 2608.10284-Lunar Reflective Interferometry
- 2610.16633-Institutional broadening of the Japanese astronomy and astrophysics research community during 1996-2025