FermiGrad-SVD: An LLM Compression Pipeline with Global Rank Selection and Gauge Fixing
cs.LG, stat.ML
Submitted: 2025-11-26
Updated: 2026-10-29
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- The Unreasonable Ineffectiveness of the Deeper Layers
- COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation
- Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives
- SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression
- ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
- Pivoting Factorization: A Compact Meta Low-Rank Representation of Sparsity for Efficient Inference in Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis