Mamba Can Learn Low-Dimensional Targets In-Context via Test-Time Feature Learning
cs.LG, stat.ML
Submitted: 2025-10-14
Updated: 2026-11-08
Terminology
Sources
- Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning
- Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions
- From Markov to Laplace: How Mamba In-Context Learns Markov Chains
- Trained Mamba Emulates Online Gradient Descent in In-Context Linear Regression
- How Can Mamba Learn In Context with Outliers and Generalize Provably?
- Gating is Weighting: Understanding Gated Linear Attention through In-context Learning
- On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning
- An Empirical Study of Mamba-based Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis