Learning Decision-Stump Thresholds in Context: Dynamics of Softmax Attention
stat.ML, cs.LG
Submitted: 2026-10-05
Updated: 2026-10-05
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Unbiased Gradients, Moving Stability Boundaries: Exact Mini-Batch Geometry in Linear Self-Attention
- Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
- The Statistical Accuracy of Neural Posterior and Likelihood Estimation
- Training Dynamics of Softmax Self-Attention: Fast Global Convergence via Preconditioning
- Transformers Can Learn Posterior Predictive Distributions In-Context
- On the Emergence of Induction Heads for In-Context Learning
- Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks
- Meta-Statistical Learning: Supervised Learning of Statistical Estimators
- Theoretical guarantees for neural estimators in parametric statistics
- Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions
- TabPFN: One Model to Rule Them All?
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Efficient quadratic entropy with distance sketches
- Embedding-Bias in Conditional Independence Testing
- From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier