SPERA: Spherical Prior EEG Foundation Model with Geometry- and Frequency-Aware Latent Prediction

arXiv:2610.10571 · cs.LG · Submitted 2026-10-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "SPERA: Spherical Prior EEG Foundation Model with Geometry- and Frequency-Aware Latent Prediction".

Jane: Detailed Research Summary of SPERA (Spherical Prior EEG Representation Architecture) SPERA is presented as a novel EEG foundation model designed for representation learning,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Let's talk about how SPERA tackles those electrode differences and frequency issues in this paper. The title itself points to the spherical prior and geometry awareness.

Jane: Exactly, Tom. They propose a Legendre-polynomial spatial prior incorporated into the attention mechanism, which they say provides a learnable pairwise geometric bias reflecting the spherical structure of EEG and applying to arbitrary montages.

Lu: That part is clever because it means the model learns how to handle different electrode geometries without being specifically trained on every single montage.

Meng: So, if I understand correctly, this prior helps the model generalize across different recording setups because it understands the underlying shape of the scalp data.

Tom: Right. And they don't stop there with just geometry; they also have this relational spectral regularizer that induces frequency-aware latents by aligning pairwise similarity structure with spectral views.

Jane: That means the latent space isn't just capturing general patterns; it’s explicitly organized around the frequencies present in EEG data, which is crucial for understanding what's happening neurally.

Lalam: So, to put it plainly, SPERA aims to create a representation that is both spatially aware of where things are on the scalp and spectrally aware of what frequencies are active.

The paper's summary: Tom: Okay, so we know they have this architecture with the Legendre prior and spectral regularizer. Now let's look at the actual pretraining they did to see how good it got.

Jane: They pretrain SPERA on a massive corpus, which is approximately eighty thousand hours of EEG data collected from twenty-nine thousand forty-eight subjects across one hundred six diverse datasets <ref:2610.10571#pg2,from 29,048 subjects across 106>. That kind of scale is pretty significant for foundation models.

Lu: That huge dataset size is what lets them achieve these results; it’s not just a small test set they're looking at, you know?

Meng: So the summary suggests that this massive pretraining allows SPERA to achieve state-of-the-art performance across nine diverse downstream tasks covering clinical, cognitive, and Brain-Computer Interface applications.

Tom: And on the performance side of things, they found that SPERA achieves the highest average balanced accuracy across all those tasks.

Jane: They specifically mention that SPERA Base surpasses the strongest baseline model called REVE Base by zero point zero three five in average balanced accuracy across those nine tasks.

Lalam: It’s not just about getting a high number on one test; it’s about this comprehensive evaluation showing how well the latent space prediction objective actually generalizes to different real-world uses for EEG.

The paper's improvements: Tom: So, beyond just achieving high accuracy, the authors point out several specific improvements they made to make SPERA work better than previous attempts.

Jane: One key improvement is the move from reconstruction-based objectives defined on the observed signal to this latent-space prediction objective within JEPA. They argue that predicting representations at the representation level is more effective than just reconstructing the raw data because you filter out some of that noise.

Lu: That's a big theoretical move; they are betting that learning what the representation *is* is better than learning what the input signal *looks like*.

Meng: On another point, they show that SSP masking generally outperforms block masking, with the best performance on each task achieved under SSP at rho approximately equal to zero point three four. That suggests preserving semantic subsequences during training helps learn more transferable EEG representations.

Tom: And they also demonstrate that the Legendre prior is better than alternative geometry encodings they tested, confirming its effectiveness in modeling the spatial aspect of EEG data across different electrode montages.

Jane: So, the improvements focus on making the latent space prediction objective stronger and ensuring those specific components—the prior and spectral regularizer—actually contribute meaningfully to capturing both shape and frequency information simultaneously.

Conclusion: Tom: Alright, we’ve walked through the architecture, seen how they pretrain it on that huge dataset, and discussed the specific improvements they made to push performance higher. It’s clear SPERA is a really comprehensive model for EEG.

Jane: Exactly. The main implication is that we can have a foundation model for EEG that is robust across different subjects and different recording setups because of the geometry-aware prior and frequency awareness from the spectral regularizer.

Lu: I think what this means for future work is that we can start thinking about how to use these learned representations to build incredibly flexible tools for analyzing neural data without needing a specific dataset for every single study.

Meng: Practically speaking, the efficiency they showed under linear probing is important because it means we can deploy these models on devices where we don't have massive computational power.

Lalam: I think this paper shows how integrating spatial geometry, frequency alignment, and latent prediction into a single JEPA framework creates a very powerful tool for handling the complexity of EEG data.

Tom: That’s the gist of it. So, SPERA is out there now as a toolkit for representation learning in EEG that's designed to be robust and efficient. That’s where we wrap up on this discussion about SPERA: Spherical Prior EEG Foundation Model with Geometry- and Frequency-Aware Latent Prediction.

Minsu Kim, Ye-Sung Kim, Hyeseong Jeon, Wooseok Hyung, Joshua Lee, *Chang-Hwan Im

Department of Electronic Engineering, Hanyang University, Seoul, Republic of Korea

cs.LG

Submitted: 2026-10-04

Updated: 2026-10-04

Comments: Accepted at NeurIPS 2026

Code: https://github.com/sees4321/SPERA_EEG

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 89/100

The gist: Detailed Research Summary of SPERA (Spherical Prior EEG Representation Architecture) SPERA is presented as a novel EEG foundation model designed for representation learning, specifically leveraging a

Key concepts

Legendre-Polynomial Spatial Prior
This is a mathematical tool used to encode the physical shape of the scalp. SPERA uses it directly in its attention mechanism to understand how different electrode positions affect brain signals. It helps the model generalize well even when presented with new or different electrode setups.
Hybrid Attention Backbone
The model uses a combination of different types of attention blocks: some handle temporal (time-based) relationships, while others handle spatial (electrode location) relationships. This hybrid structure allows the model to efficiently capture both how brain activity changes over time and how it relates across different scalp points.
Relational Spectral Regularizer
This component forces the learned internal representations of EEG data to align with known frequency information derived from the raw signals. It ensures that the latent space captures meaningful spectral details, helping the model understand which brain frequencies are important for a given task.

Terminology

Summary

Detailed Research Summary of SPERA (Spherical Prior EEG Representation Architecture)

SPERA is presented as a novel EEG foundation model designed for representation learning, specifically leveraging a joint-embedding predictive architecture (JEPA) to predict representations within a learned latent space. The core innovation of SPERA lies in its multi-faceted architectural design, which integrates spatial prior encoding, hybrid attention mechanisms, and spectral regularization to effectively capture the complexities inherent in electroencephalography (EEG) data across heterogeneous datasets and electrode montages.

Core Architectural Innovations

SPERA's architecture is built upon several key components that adapt the general JEPA framework to the specific characteristics of EEG:

  1. Legendre-Polynomial Spatial Prior: A crucial element introduced by SPERA is the incorporation of a Legendre-polynomial spatial prior directly into the attention mechanism. This prior is specifically engineered to encode and account for varying scalp electrode geometries, ensuring that the model can generalize effectively across different electrode montages. Analysis of the learned coefficients reveals that this prior heavily favors low Legendre degrees (1–3), indicating that the learned bias is dominated by broad, smoothly varying angular structures on the scalp rather than fine-grained local details.

  2. Hybrid Attention Backbone: The model employs a sophisticated attention structure consisting of factorized temporal and spatial attention blocks, which are interleaved with periodic full-attention blocks. This hybrid approach is designed to efficiently capture both the temporal dynamics and spatial relationships within EEG signals.

  3. Relational Spectral Regularizer: To induce frequency-aware latents, SPERA utilizes a relational spectral regularizer. This component aligns the similarity structure of the learned latent representations with explicit spectral views, ensuring that the latent space captures meaningful frequency information derived from EEG data.

Pretraining and Performance Metrics

SPERA is pretrained on an extensive corpus comprising approximately 80,000 hours of EEG data collected from 29,048 subjects across 106 diverse datasets. This massive pretraining enables the model to achieve state-of-the-art performance:

  • Superior Accuracy: SPERA achieves the highest average balanced accuracy across nine diverse downstream tasks spanning clinical, cognitive, and Brain-Computer Interface (BCI) applications. Specifically, SPERA-Base surpasses the strongest baseline (REVE-Base) by 0.035 in average balanced accuracy.

  • Robustness and Efficiency: The model demonstrates strong parameter efficiency under linear probing and exhibits significant robustness across varying recording conditions. Furthermore, it achieves the smallest across-seed standard deviation in balanced accuracy (plus or minus 0.0018) among compared methods.

  • Scalability: Scaling analysis confirms its efficiency, showing that SPERA-Base outperforms REVE-Large under linear probing despite utilizing less than half the parameters.

Ablation Studies and Theoretical Support

Controlled ablation studies provide strong evidence supporting the complementary nature of SPERA's components:

  • Latent Prediction vs. Reconstruction: These experiments demonstrate that latent prediction outperforms a matched reconstruction objective, validating the core hypothesis of predicting in latent space rather than just reconstructing raw data.

  • Prior Efficacy: The Legendre prior is shown to outperform alternative geometry encodings evaluated, confirming its effectiveness in spatial modeling.

  • Synergistic Components: Overall results strongly support the synergy between latent-space prediction, geometry-aware spatial modeling (via the Legendre prior), and frequency-aware supervision (via the spectral regularizer) as essential components for robust EEG representation learning across different datasets and electrode montages.

Data Efficiency and Robustness Evaluation

Beyond peak accuracy, SPERA exhibits strong data efficiency and robustness in downstream applications:

  • Few-Shot Learning: Experiments on BCIC-IV-2a, PhysioNet-MI, and ISRUC using linear probing (LP) with frozen encoders show that SPERA provides highly efficient representations. For instance, on BCIC-IV-2a, SPERA-Base surpassed REVE-Base using only 10% of the training data. On ISRUC, it exceeded the reference with just 2% of the data.

  • Noise Robustness: When evaluated under additive Gaussian noise (on TUAB, ISRUC, and PhysioNet-MI), SPERA-Small generally maintains higher mean balanced accuracy than comparison models. Notably, the JEPA variant maintained a higher mean accuracy under corruption conditions compared to matched JEPA or reconstruction variants.

  • Artifact Robustness: Ocular artifact robustness evaluations on PhysioNet-MI and TUAB showed that SPERA-Small achieved the highest mean balanced accuracy on TUAB, while JEPA generally outperformed reconstruction under injected artifact conditions.

  • Representation Structure: UMAP visualization indicated that raw EEG samples lacked clear class structure, but frozen pretrained features already captured task-related structure to varying degrees (e.g., ISRUC sleep stages showed distinguishable clusters even without task supervision). Fine-tuning further enhanced the separability of these clusters across all three datasets.

Conclusion and Contribution

SPERA is a highly comprehensive EEG foundation model that successfully integrates spatial geometry encoding, temporal/spatial attention mechanisms, and spectral regularization within a JEPA framework. Its extensive pretraining on diverse data yields superior performance across multiple clinical and cognitive tasks while maintaining strong parameter efficiency. Crucially, SPERA demonstrates not only high predictive accuracy but also excellent data efficiency in few-shot settings and robustness against common real-world challenges such as Gaussian noise and ocular artifacts. The paper provides a complete toolkit, including the architecture, training procedures, hyperparameters, and code for training and evaluation.

Improvements for AI systems

  1. Bold header: Latent-space prediction for EEG

This core architecture adopts the joint-embedding predictive architecture (JEPA) to predict in latent space, allowing the model to learn representations by predicting latent representations of masked target regions from visible context, which is more effective than reconstruction-based objectives defined on the observed signal.

  1. Bold header: Geometry-aware spatial modeling

The introduction of a Legendre polynomial spherical prior, incorporated into attention to encode varying scalp electrode geometries provides a learnable pairwise geometric bias that reflects the spherical structure of scalp EEG and applies to arbitrary montages. This enables the model to be dataset-agnostic regarding electrode configurations.

  1. Bold header: Frequency-aware latent representations

The paper introduces a relational spectral regularizer (RSReg) that aligns the pairwise similarity structure of the predicted representations with that of the corresponding spectral targets, which induces frequency-aware latents by aligning pairwise similarity structure with spectral views. This allows for encoding frequency information into the latent space without requiring absolute spectrum reconstruction.

  1. Bold header: Hybrid attention backbone

The model utilizes a hybrid backbone of factorized attention blocks interleaved with periodic full-attention blocks for cross-axis interactions, which addresses the limitation where purely factorized attention restricts direct cross-axis interaction. This design allows the model to capture both temporal and spatial dependencies effectively across varying channel montages.

  1. Bold header: Efficient parameter scaling

The study demonstrates that SPERA exceeds REVE-Large under LP despite using less than half the parameters, showing that latent-space prediction is highly scalable, with SPERA-Base (194M) exceeding REVE-Large (408M) by +0.013 under LP.

  1. Bold header: Robustness to masking strategies

The analysis shows that SSP masking generally outperforms block masking, with the best performance on each task achieved under SSP at ρ ≈ 0.34, suggesting that preserving semantic subsequences is beneficial for learning transferable EEG representations.

  1. Bold header: Anatomical localization and attribution

Using Integrated Gradients (IG), the model's electrode-level attributions align with known EEG neurophysiological patterns, showing a shared profile across tasks, which can be used to map specific electrode contributions to established cortical regions like the sensorimotor strip for motor imagery.

Abstract

Electroencephalography (EEG) provides a non-invasive measure of ongoing neural activity, but building general-purpose EEG models remains challenging due to the heterogeneity of subjects, devices, and electrode montages. Existing EEG foundation models predominantly rely on reconstruction-based objectives defined on the observed signal, which contains both neural and non-neural components. We introduce SPERA (Spherical Prior EEG Representation Architecture), an EEG foundation model that adopts the joint-embedding predictive architecture (JEPA) to predict in latent space. SPERA introduces a Legendre-polynomial spatial prior, incorporated into attention to encode varying scalp electrode geometries. Two further components adapt the model to EEG: factorized temporal and spatial attention interleaved with periodic full-attention blocks, and a relational spectral regularizer aligning latent similarity structure with spectral views. Pretrained on approximately 80,000 hours of EEG from 29,048 subjects across 106 datasets, SPERA achieves the highest average balanced accuracy across nine downstream tasks spanning clinical, cognitive, and BCI applications. SPERA further exhibits strong parameter efficiency under linear probing and robustness across varying recording conditions, suggesting its potential as a general-purpose backbone for diverse EEG analyses.

Sources

Related papers