The Noise Floor of a Quantization Comparison: Engine, restart, batch and kernel drift, measured against 8-bit and 4-bit effects in two decision models
cs.LG
Submitted: 2026-11-11
Updated: 2026-11-11
Code: https://github.com/fla-org/flash-linear-attention
Terminology
Sources
- Accuracy is Not All You Need
- Which Decisions Low-Bit Quantization Breaks, and How to Predict Them
- What We Observe as LLM Behavior Can Be a Side-effect of Inference Backend
- The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility
- Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference
- Flide: Accelerating Decision Inference with Exact State Forking
- Same Text, Different Prediction: Serving-Context Nondeterminism in Text Classifiers
- Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving
- Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
- How Far Does Second-Order Linear Response Predict Quantization Mismatch? Pre-Registered Measurements on Five Language Models
- Efficient Intent Detection with Dual Sentence Encoders
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Measuring Massive Multitask Language Understanding
- ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction
- Adversarial NLI: A New Benchmark for Natural Language Understanding
- RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis