Pre-registered tests of solid-state-physics-inspired LLM compression: a cluster-level negative result at small-language-model scale
cond-mat.dis-nn, cond-mat.mtrl-sci, cond-mat.other, cs.LG
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/zhenyi4/ssa
Sources
- SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
- Deep learning and the renormalization group
- Pruning Deep Neural Networks via a Combination of the Marchenko-Pastur Distribution and Regularization
- Attention Degradation, Function Token Anchoring, and the Limits of Attention-Based Intervention in Large Language Models
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Recency Biased Causal Attention for Time-series Forecasting
- Pre-registration for Predictive Modeling
- Tensorized Embedding Layers for Efficient Model Compression
- Language model compression with weighted low-rank factorization
- Compressing Transformer Language Models via Matrix Product Operator Decomposition: A Case Study on PicoGPT
- Kimi Linear: An Expressive, Efficient Attention Architecture
- Neural Network Renormalization Group
- Layer Collapse Can be Induced by Unstructured Pruning
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators
- MoBA: Mixture of Block Attention for Long-Context LLMs
- An exact mapping between the Variational Renormalization Group and Deep Learning
- Dynamical Mean-Field Theory of Self-Attention Neural Networks
Related papers
- Probing critical phases in quasiperiodic systems via subsystem information capacity
- Few-Shot Neuromorphic Vision in a Nonlinear Photonic Network Laser
- Hyperbolic lattices with mass disorder: Phases and phase transitions
- Non-Markovian dephasing of disordered, quasi-one-dimensional fermion systems
- Adaptive Neural Quantum States: A Recurrent Neural Network Perspective
- Machine learning Majorana topology using unsupervised and supervised learning