CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs
cs.CL
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/SAI-Lab-NYU/CodeQuant2
Terminology
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- Training Verifiers to Solve Math Word Problems
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
- Soft Adaptive Policy Optimization
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- SqueezeLLM: Dense-and-Sparse Quantization
- Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform
- ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
- Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models
- SpinQuant: LLM quantization with learned rotations
- ReFT: Reasoning with Reinforced Fine-Tuning
- LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Massive Activations in Large Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotation
- Interleaved Reasoning for Large Language Models via Reinforcement Learning
- Qwen3 Technical Report
- CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data