Lost in the bf16 Cast: Exporting Ternary Language Models Can Revert Most Low-Learning-Rate Code Changes

cs.LG, cs.CL

Submitted: 2026-10-06

Updated: 2026-10-14

Comments: 14 pages, 4 figures, 17 tables

Code: https://github.com/microsoft/BitNet

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Sources

• BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
• Training Verifiers to Solve Math Word Problems
• Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization
• Measuring Mathematical Problem Solving With the MATH Dataset
• Tequila: Trapping-free Ternary Quantization for Large Language Models
• A Study of BFLOAT16 for Deep Learning Training
• Synaptic Metaplasticity in Binarized Neural Networks
• Scheduling Weight Transitions for Quantization-Aware Training
• FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization
• Scale-QLoRA: Code-Invariant Adapter Merging for Native 4-bit Microscaling LLMs
• Oscillation-free Quantization for Low-bit Vision Transformers
• In-Cell Learning: Language Models That Update Their Own Weights in Sequence Without Changing the File They Ship
• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
• BitNet b1.58 2B4T Technical Report
• Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL
• Up or Down? Adaptive Rounding for Post-Training Quantization
• Overcoming Oscillations in Quantization-Aware Training
• ECO: Quantized Training without Full-Precision Master Weights
• TerMeZO: Ternary Sparse Zeroth-Order Optimization for Fine-tuning BitNet Models at the Edge
• Fine-Tuning Low-Bit Models with Gradient in Quantized Code Space

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks