Where Should a Gate Look? Input vs. State Gating in Diagonal State Space Models
cs.LG, cs.AI
Submitted: 2026-11-08
Updated: 2026-11-08
Code: https://github.com/RiccardoZattra/Where-Should-a-Gate-Look
Terminology
Sources
- Were RNNs All We Needed?
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Context-Selective State Space Models: Feedback is All You Need
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis