Scaling subjects in cross-modal alignment: video decoding with EEG foundation model
q-bio.NC
Submitted: 2026-10-07
Updated: 2026-10-07
License: http://creativecommons.org/licenses/by/4.0/
The gist: Naturalistic visual decoding from EEG has long been constrained by cohort size: existing scaling literature caps out at a few dozen subjects, leading prior work to conclude that scaling subject
Terminology
Abstract
Naturalistic visual decoding from EEG has long been constrained by cohort size: existing scaling literature caps out at a few dozen subjects, leading prior work to conclude that scaling subject cohorts yields minimal performance gains. Training a cross-modal EEG--video contrastive encoder on a cohort larger by more than an order of magnitude, we find the axis is productive but has an onset. Below S about50 --- the entirety of the range prior work occupies --- no model improves meaningfully over an untrained encoder; above it, decoding rises log-linearly in subject count with no saturation at the top of our ladder, on a fitted stimulus-feature probe and on fit-free movie-moment retrieval alike, and the gain transfers to a second, unseen film. How far the axis carries then depends on initialisation more than on capacity: an encoder initialised from an EEG foundation model scales about 1.6 times faster per doubling of the cohort than randomly initialised encoders at two depths, is the only one still converting subjects into retrieval accuracy at the top of the ladder, and converges on a fraction of the alignment compute. Subject scaling on shared naturalistic stimuli is thus an effective and currently unsaturated frontier for scaling EEG foundation models.
Sources
- EEGDash: An open-source platform for machine learning on public neurophysiological data
- V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
- Scaling laws for decoding images from brain activity
- NeuralBench: A Unifying Framework to Benchmark NeuroAI Models
- Temporal Context and Architecture: A Benchmark for Naturalistic EEG Decoding
- CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processing
- Decoding Natural Images from EEG for Object Recognition
- TransNet V2: An effective deep network architecture for fast shot transition detection
Related papers
- BrainWave: A Brain Signal Foundation Model for Clinical Applications
- Toward Robust, Reproducible, and Widely Accessible Intracranial Speech Brain-Computer Interfaces: A Comprehensive Narrative Review of Neural Mechanisms, Hardware, Algorithms, Evaluation, Clinical Pathways and Future Directions
- CytoNet: A Foundation Model for the Human Cerebral Cortex at Cellular Resolution
- Decoding Alignment without Encoding Alignment: A critique of similarity analysis in neuroscience
- Emergence of psychopathological computations in large language models
- NeuroAI and Beyond: Bridging Between Advances in Neuroscience and Artificial Intelligence