Two-Level Softmax Sampling Done Right: Correcting Bias from Size Imbalance and Dispersion

cs.LG, cs.IR, stat.ML

Submitted: 2026-10-07

Updated: 2026-10-07

Comments: NeurIPS 2026

Code: https://github.com/bendadaw/neurips-2ls

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Sources

• Adaptive Sampled Softmax with Inverted Multi-Index: Methods, Theory and Applications
• Doubly Sparse: Sparse Mixture of Sparse Experts for Efficient Softmax Inference
• CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference
• VQ-Logits: Compressing the Output Bottleneck of Large Language Models via Vector Quantized Logits
• Efficient Item ID Generation for Large-Scale LLM-based Recommendation
• FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference
• DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Models

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks