Fast and accurate false-alarm probabilities for linear detection statistics in Poisson noise
summary
The gist
Fast and accurate false-alarm probabilities for linear detection statistics in Poisson noise is a new method for computing false-alarm probabilities and detection thresholds for arbitrary linear…
In short
This method provides a fast and accurate way to calculate false-alarm probabilities for linear detection statistics when background noise follows Poisson statistics. It treats the detection statistic as a compound Poisson random variable, allowing computation via FFT or Panjer recursion instead of slow simulations. This enables reliable estimation of very low false-alarm probabilities.
Key concepts
- Compound Poisson Random Variable
- This is a random variable formed by summing a random number of independent, identically distributed variables. In this context, it means the total detection statistic (S) is the sum of individual photon contributions, where the *number* of those contributions itself follows a Poisson distribution.
- Fast Fourier Transform (FFT) Implementation
- This computational strategy uses a single Fourier transform to efficiently calculate the probability distribution PH0(s). It involves calculating the characteristic function of one-photon noise, applying an exponential function, and then using an inverse FFT to recover the final probability distribution. This method is highly efficient.
- Panjer Recursion Implementation
- This is an alternative computational technique that uses a recursive formula to determine the probability distribution. It avoids potential numerical errors like catastrophic cancellations found in some Fourier transform methods, offering a computationally lighter approach with complexity O(Npix + NgridNpix).
- Linear Detection Statistic
- This refers to any detection measurement derived from a linear combination of observed events. The method is versatile because it works for *any* such statistic, regardless of the specific filter used, as long as the underlying noise follows Poisson statistics.
Terminology used across episodes
This episode discusses
- Fast and accurate false-alarm probabilities for linear detection statistics in Poisson noise · Paper Radio
- An Introduction to L'evy and Feller Processes. Advanced Courses in Mathematics - CRM Barcelona 2014
- X-Sifter: detecting transients in X-ray data using the optimal Poisson matched filter
The paper
Fast and accurate false-alarm probabilities for linear detection statistics in Poisson noise · Read on arXiv
Department of Physics, Bar-Ilan University · Lawrence Berkeley National Laboratory · Benoziyo Center for Astrophysics, Weizmann Institute of Science
Transcript
Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.
Vera: We're looking at the paper: "Fast and accurate false-alarm probabilities for linear detection statistics in Poisson noise".
Jocelyn: Here's the gist of the paper. Fast and accurate false-alarm probabilities for linear detection statistics in Poisson noise is a new method for computing false-alarm probabilities and detection thresholds for arbitrary linear filters in low-count regimes where background noise follows Poisson statistics, which avoids computationally expensive simulations The Core Finding The method derives an accurate and efficient solution for computing false-alarm probabilities for arbitrary linear detection statistics under Poisson noise The detection statistic is shown to be a compound Poisson random variable, which allows for efficient implementations based on the Fast Fourier Transform (FFT) and the Panjer recursion This approach transforms the computation of detection significance under Poisson noise from a simulation problem into a deterministic numerical calculation Statistical Formulation The statistic S is defined as a compound Poisson random variable, which is the sum of independent identically distributed (i.i.d.) random variables where the number of terms in the sum is itself Poisson-distributed The probability distribution of S under the null hypothesis H0, denoted as PH0(s), can be expressed using a known result from Feller (1968) This distribution is given by Equation 16, which involves the Poisson probability Ppois(n) and the n-fold convolution of the one-photon probability distribution pX with itself The single-photon distribution pX is determined by Equation 18, which depends on the spatial distribution of background photons and kernel values Computational Implementations The paper presents two primary computational strategies for evaluating the compound Poisson distribution The FFT Implementation Using the Characteristic Function involves calculating the characteristic function of the one-photon contribution X using a single Fourier transform, followed by a pointwise complex exponential, and finally an inverse Fourier transform to recover PH0(s).
Vera: This method has an overall computational complexity of O (Npix + Nω log Nω), where Nω is the FFT length.
Jocelyn: The implementation includes adaptive refinement procedures to ensure stability against discretization and aliasing approximations.
Vera: The Panjer-Recursion Implementation offers an alternative, utilizing a recursive formula that avoids catastrophic cancellations in the inverse Fourier transform, with a complexity of O(Npix + NgridNpix).
Title and authors: Jocelyn: So let’s look at who wrote this paper. The authors are Maayane T Soumagnac, Eran O Ofek, Shachar Israeli and David Glukhov.
Vera: They’re all working in different areas of astrophysics, which is typical for these kinds of papers. We see Vera here focusing on survey photometry and spectra, while Jocelyn deals with pulsar and time-domain radio astronomy.
Subrahmanyan: I'm interested in how they bridge the gap between those observational constraints—like the kernel choice Stetson did—and the statistical calculation itself.
Jocelyn: They’re addressing two specific challenges: first, picking a kernel K that makes an optimal detection statistic S, and second, calculating that threshold above which we can declare a detection.
Vera: It seems they are tackling the mathematical complexity of the low-count regime where the Gaussian approximation for these statistics just doesn't hold up anymore.
Subrahmanyan: That’s crucial because if those approximations fail, our models for how structure forms or how signals propagate get skewed.
Jocelyn: The title itself suggests they are focused on speed and accuracy in low count regimes, which is where most of us spend a lot of time chasing faint signals.
Vera: It really feels like they are providing a tool that can handle the extreme false alarm probabilities that simulations usually make impractical to estimate.
Subrahmanyan: That capability would be useful for testing new detection methods in X-ray or neutrino detectors, where those low count environments are common.
The paper's summary: Vera: So, summarizing the core finding of this work, they prove that the detection statistic is indeed a compound Poisson variable. This means its probability distribution is known and has been studied extensively in risk theory for calculating aggregate loss distributions.
Jocelyn: And once you know that it’s compound Poisson, you can use established methods to compute this probability distribution rather than running huge Monte Carlo simulations every single time.
Subrahmanyan: That shifts the computational burden from brute-force simulation to a deterministic numerical calculation, which is a massive win for efficiency.
Vera: They then show two main ways to actually perform this computation efficiently: one based on the Fast Fourier Transform and another using a Panjer recursion.
Jocelyn: The FFT approach uses the characteristic function of the one photon contribution, applying a single Fourier transform, then a complex exponential, and finally an inverse Fourier transform to get back the probability distribution.
Subrahmanyan: And for that FFT method, they give us a complexity bound of O (Npix plus Nomega log Nomega), where Nomega is related to the FFT length.
Vera: The Panjer recursion is another path, which uses a recursive formula that avoids some of the numerical issues you might run into with the inverse Fourier transform.
Jocelyn: That recursive method has a complexity of O (Npix plus NgridNpix), and it lets you compute up to a bound nmax determined by how much tolerance you set for your count.
The paper's improvements: Vera: The real improvement they present is the choice of these two computational strategies—FFT versus Panjer recursion—which both offer significant speed gains over simulation methods.
Jocelyn: They also present an alternative approach in Appendix A, which is based on enumerating all possible realizations of the occupancy vector Mb, which they say is equivalent to the compound variable formalism.
Subrahmanyan: But they admit that this enumeration method is computationally heavier than the FFT or Panjer methods.
Vera: The paper then validates these results through five consistency checks comparing different discretization levels and aliasing tolerances, showing stability when you refine the binning or make the aliasing tolerance stricter.
Jocelyn: That stability check is important because it confirms that whether you use the FFT or Panjer, you’re getting a reliable threshold estimate.
Subrahmanyan: It also highlights a practical limitation they flag: for the FFT approach, there's a potential loss of numerical accuracy in the extreme tail of the distribution because of catastrophic cancellation during that inverse Fourier transform.
Conclusion: Vera: So to wrap up, this paper shows that by recognizing the detection statistic as a compound Poisson variable, we get an accurate and efficient way to compute false alarm probabilities for arbitrary linear filters in low count regimes.
Jocelyn: The FFT and Panjer methods are the key computational tools here, offering speeds in milliseconds compared to seconds for simulations.
Subrahmanyan: From my side, this method's main impact is enabling us to reliably evaluate extremely low false alarm probabilities that are often impractical to estimate by simulation alone.
Vera: It means we can finally get a handle on thresholds corresponding to significance levels that would otherwise be computationally inaccessible for source detection in low-count X-ray images or other faint sources.
Jocelyn: The Python and MATLAB implementations are available, which should help researchers bring this into use for their own work quickly.
Subrahmanyan: It’s a powerful tool because it’s independent of the specific linear filter you choose, so we can apply this framework broadly to any problem involving linear statistics and Poisson-distributed events.
Vera: That's it for this paper on "Fast and accurate false-alarm probabilities for linear detection statistics in Poisson noise." We’ll be looking at some papers next.
More episodes
- 2607.17929-gevolution 2.0: GPU-accelerated relativistic N-body simulations for cosmology
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2509.02657-On the synergetic use of Ariel and JWST for exoplanet atmospheric science
- 2609.10042-Modelling Fourier filtering wavefront sensors for PSD-based methods: the AOPERA tool
- 2511.05639-Performance of the AstroPix Prototype Module for the Barrel Imaging Calorimeter at the ePIC Detector and in Space-Based Payloads
- 2610.19909-RPCA: a Resolved PSF reconstruction method for undersampled images
- 2512.03440-An Analysis of LIGO Glitches Using t-SNE During the First Part of the Fourth LIGO-Virgo-KAGRA Observing Run
- 2608.10284-Lunar Reflective Interferometry
- 2610.16633-Institutional broadening of the Japanese astronomy and astrophysics research community during 1996-2025