How Much Does Random Rotation Help GPTQ? Sharp Worst-Case Rate Gaps in the High-Rate Regime
cs.IT, cs.LG, math.IT
Submitted: 2026-11-01
Updated: 2026-11-01
Terminology
Sources
- QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
- On the role of MMSE estimation in approaching the information-theoretic limits of linear Gaussian channels: Shannon meets Wiener
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- The Llama 3 Herd of Models
- Transforms for LLM Quantization: The Great Inversion and Format Co-Design
- Revisiting Error Feedback for Model Quantization
- WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization
- Pointer Sentinel Mixture Models
- Optimal Quantization for Matrix Multiplication
- High-Rate Quantized Matrix Multiplication I
- High-Rate Quantized Matrix Multiplication II
- Equality of Lyapunov and stability exponents for products of isotropic random matrices
- D-Quant: Driftable Entropy Coding for KV Cache Quantization
- QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
- Introduction to the non-asymptotic analysis of random matrices
- OJBKQ: Objective-Joint Babai-Klein Quantization
- Qwen3 Technical Report
- Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronos
Related papers
- The Gaussian Multiuser Channel Capacity Region
- The mathematical landscape of partial information decomposition: A comprehensive review of properties and measures
- Electromagnetic Signal and Information Theory: A Continuous-Aperture Array Perspective
- Analytically Characterized Optimal Power Control for Signal-Level-Integrated Sensing, Computing and Communication in Federated Learning
- Information Spectrum Methods for Multiple-Access Channels--Part I: epsilon-Capacity Regions in the Mixed Gaussian Case
- Seed Recycling: Explicit Codes for Mixed Synchronization Errors Matching the Leading Term of the Gilbert--Varshamov Bound