Lost in Rounding: bf16 Recurrent State Makes Hybrid Reasoning Models Think Longer
cs.CL, cs.AI
Submitted: 2026-11-09
Updated: 2026-11-09
Code: https://github.com/sgl-project/sglang
Terminology
Sources
- Stochastic Rounding in Low-Precision Transformer Inference: A Variable-Precision Emulation Study of a Small GPT-2
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- Mind the Cap: Output-Budget Regimes Change the Measured Multilingual Reasoning Gap
- Deep Learning with Limited Numerical Precision
- Measuring Mathematical Problem Solving With the MATH Dataset
- Low-Bit GDN Recurrent-State Quantization: Global Gradient-aware Allocation and Pure-Write Sliding Windows
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- Low-Discrepancy Dither for Quantized Recurrent State Caches
- A KL Lens on Quantization: Fast, Forward-Only Sensitivity for Mixed-Precision SSM-Transformer Models
- Preserving Updates in Low-Bit Recurrent State Quantization
- Let's Verify Step by Step
- ReplaySSM: Cache SSM Inputs, Not State
- Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
- Quantizing Recurrent Language Models Is Easier Than You Think
- The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits
- NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
- Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI
- Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
- Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
- LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data