Shannon observables: A scalable approach to information decomposition
cs.IT, cs.AI, cs.LG, math.IT, nlin.AO, physics.data-an
Submitted: 2025-04-22
Updated: 2026-11-11
Comments: 28 pages, 8 figures, 3 tables. v2: substantially revised and retitled (formerly "Shannon invariants"); adds an analysis of error-correcting codes, an extension to directed information flow with a cortical atlas from MEG recordings, single-neuron lesion and dropout experiments, and bounds linking vulnerability to performance loss after neuron removal
Code: https://github.com/Priesemann-Group/nninfo
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Quantifying Hierarchical Selection
- An information-theoretic approach to hypergraph psychometrics
- Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks
- Interpretable Diffusion via Information Decomposition
- Nonnegative Decomposition of Multivariate Information
- Towards an extended taxonomy of information dynamics via Integrated Information Decomposition
- The Fast M\"obius Transform: An algebraic approach to information decomposition
- Software in the natural world: A computational approach to hierarchical emergence
- Characterising directed and undirected metrics of high-order interdependence
- THOI: An efficient and accessible library for computing higher-order interactions enhanced by batch-processing
- Null models for comparing information decomposition across complex systems
Related papers
- The Gaussian Multiuser Channel Capacity Region
- The mathematical landscape of partial information decomposition: A comprehensive review of properties and measures
- Electromagnetic Signal and Information Theory: A Continuous-Aperture Array Perspective
- Analytically Characterized Optimal Power Control for Signal-Level-Integrated Sensing, Computing and Communication in Federated Learning
- Information Spectrum Methods for Multiple-Access Channels--Part I: epsilon-Capacity Regions in the Mixed Gaussian Case
- Seed Recycling: Explicit Codes for Mixed Synchronization Errors Matching the Leading Term of the Gilbert--Varshamov Bound