Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting
cs.LG, cs.AI, cs.SI
Submitted: 2026-09-26
Updated: 2026-09-26
Comments: Accepted by NeurIPS 2026 as a spotlight paper
Code: https://github.com/keris-yang/Spectral-Reversal
License: http://creativecommons.org/licenses/by/4.0/
The gist: Pre-training Graph Neural Networks (GNNs) via self-supervised learning has become a dominant paradigm, yet efficiently adapting frozen encoders remains a challenge.
Terminology
Abstract
Pre-training Graph Neural Networks (GNNs) via self-supervised learning has become a dominant paradigm, yet efficiently adapting frozen encoders remains a challenge. Graph prompting offers a parameter-efficient alternative to fine-tuning, but existing methods largely treat pre-trained models as opaque feature extractors, ignoring their internal spectral structure. In this work, we identify a systematic phenomenon in pre-trained GNNs, which we term spectral bias: optimization during pre-training disproportionately aligns representations with directions associated with large singular values, leaving low-energy directions under-explored. We show that these underutilized directions can encode complementary information that is beneficial for downstream adaptation, especially under distribution shift. To leverage this insight, we propose Spectral Reverse Prompt (SRP), a prompting framework that rebalances the spectral contributions of frozen GNN encoders. SRP applies a learnable soft-thresholding mask in the spectral domain to down-weight dominant directions while amplifying weaker ones. In addition, SRP incorporates a null-space augmentation module that captures variation in directions with minimal activation under the frozen encoder. Extensive experiments across multiple benchmarks demonstrate that SRP achieves state-of-the-art performance with minimal additional parameters, highlighting that reweighting spectral components is a principled and effective strategy for parameter-efficient graph adaptation.
Sources
- Graph Prompt Learning: A Comprehensive Survey and Beyond
- Deep Graph Contrastive Representation Learning
- SCGFM-ART: Amortized Relational Transport for Structure-Centric Graph Foundation Models
- Dynamic Graph Prompting via Topology-Routed Mixed-Curvature Experts
- TUDataset: A collection of benchmark datasets for learning with graphs
- ProG: A Graph Prompt Learning Benchmark
- The Post-GCN Decade Revisited: Curvature-Stratified Evaluation of Relational Learning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks