The Degeneracy Distillery
T. Lucas Makinen, Deaglan J. Bartlett, Niall Jeffrey, Benjamin D. Wandelt
cs.LG, astro-ph.IM, physics.comp-ph, physics.data-an, stat.ML
Submitted: 2026-06-22
Comments: 30 pages, 10 figures. Supporting code found at https://github.com/tlmakinen/degeneracy_distillery
Code: https://github.com/tlmakinen/degeneracy_distillery
License: http://creativecommons.org/licenses/by/4.0/
The gist: When two or more parameters or labels produce similar data, they are degenerate, or hard to distinguish.
Terminology
Abstract
When two or more parameters or labels produce similar data, they are degenerate, or hard to distinguish. Degeneracies render both label prediction and inverse problems difficult, since both machine learning algorithms and probabilistic samplers rely on the distinguishability of data and its gradients with respect to parameters. However, identifying degeneracies in physical models or real-world datasets can be elucidating about the choice of model or the underlying process that produces the data. We present the degeneracy distillery, a method that (1) detects and (2) resolves degenerate parameter combinations (a) automatically and (b) symbolically, from parameter-data (or parameter-simulation) pairs alone, through estimation and flattening of the Fisher information matrix. By exploring the information geometry of the likelihood, we characterize degeneracies as an intrinsic property of the physical model, requiring no realised data observation. We demonstrate our approach on a range of synthetic and real-world problems, discovering symbolic coordinate transformations that identify the combinations of parameters of a model which yield independent effects on the data. The resulting coordinates flatten the Fisher information in expectation globally, in contrast to posterior-based methods that flatten only at a single point, and substantially reduce the simulation budget required for downstream neural posterior estimation. In test cases we require up to 10 times fewer simulations for posterior estimation at matched validation calibration whilst simultaneously gaining physical insight on the system.
Sources
- On the Parameter Combinations That Matter and on Those That do Not
- LtU-ILI: An All-in-One Framework for Implicit Inference in Astrophysics and Cosmology
- Sampling-Based Accuracy Testing of Posterior Estimators for General Inference
- pmwd: A Differentiable Cosmological Particle-Mesh $N$-body Library
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks