SAMBA: Toward an EEG Foundation Model via Differential Mamba and Coordinate-Driven Spatial Embedding
cs.LG
Submitted: 2025-11-23
Updated: 2026-11-03
Comments: Accepted at the NeurReps 2026 Proceedings Track. 21 pages
Code: https://github.com/Jiazhen-Hong/SAMBA
License: http://creativecommons.org/licenses/by/4.0/
The gist: Modeling long-context electroencephalography (EEG) signals is critical for learning generalizable EEG representations…
Terminology
Abstract
Modeling long-context electroencephalography (EEG) signals is critical for learning generalizable EEG representations, as EEG recordings are acquired at high sampling rates and often span extended durations that capture evolving neural states beyond short task- specific windows. Transformer-based models have shown promise in modeling short sequences of a few seconds; however, their quadratic complexity limits scalability to longer contexts. Moreover, substantial variability in electrode montages across datasets and recording devices poses a major challenge for building EEG models that generalize across domains. We propose SAMBA, a self-supervised learning framework with a Mamba- based U-shaped encoder-decoder architecture, which effectively captures long-range temporal dependencies and montage variability in EEG data. Leveraging the inherent ability of Mamba in processing long context sizes, we introduce: (1) Temporal Semantic Random Masking for semantic-level sequence reconstruction, (2) a Multi-Head Differential Mamba module to suppress redundancy and emphasize salient temporal structures, and (3) a Coordinate-Driven Spatial Embedding that learns unified embeddings in a three- dimensional Euclidean space, enabling robustness across devices. Experiments on thirteen EEG datasets across diverse tasks, electrode configurations, and sequence durations demonstrate that SAMBA achieves competitive or improved performance compared to recent state-of-the-art methods, while maintaining low memory consumption and efficient inference. The learned spatial weight maps exhibit close alignment with task-relevant neurophysiological regions, highlighting the interpretability of SAMBA. The code is available at: https://github.com/Jiazhen-Hong/SAMBA
Sources
- MAEEG: Masked Auto-encoder for EEG Representation Learning
- Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI
- Efficiently Modeling Long Sequences with Structured State Spaces
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- Differential Transformer
- Transformer-based Spatial-Temporal Feature Learning for EEG Decoding
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Efficient Long Sequence Modeling via State Space Augmented Transformer
- Zamba: A Compact 7B SSM Hybrid Model
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis