How Far Does Second-Order Linear Response Predict Quantization Mismatch? Pre-Registered Measurements on Five Language Models
cs.CL, cs.CR, cs.LG
Submitted: 2026-11-09
Updated: 2026-11-09
Terminology
Sources
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
- HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
- Extreme Compression of Large Language Models via Additive Quantization
- Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning
- QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models
- Deep Learning with Limited Numerical Precision
- SqueezeLLM: Dense-and-Sparse Quantization
- Limitations of the Empirical Fisher Approximation for Natural Gradient Descent
- BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- Up or Down? Adaptive Rounding for Post-Training Quantization
- OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
- WoodFisher: Efficient Second-Order Approximation for Neural Network Compression
- Massive Activations in Large Language Models
- QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
- SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
- HAWQV3: Dyadic Neural Network Quantization
- The Super Weight in Large Language Models
- Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data