Spectral Ghost in Representation Learning: A Unified View from Component Analysis to Self-Supervised Learning
cs.LG
Submitted: 2026-01-28
Updated: 2026-11-10
Terminology
Sources
- Self-labelling via simultaneous clustering and representation learning
- A Cookbook of Self-Supervised Learning
- VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- On the duality between contrastive and non-contrastive self-supervised learning
- DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
- word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
- Self-supervised Pretraining of Visual Features in the Wild
- Representation Learning via Non-Contrastive Mutual Information
- Learning deep representations by mutual information estimation and maximization
- Primal-Dual Spectral Representation for Off-policy Evaluation
- $f$-MICL: Understanding and Generalizing InfoNCE-based Contrastive Learning
- Offline Imitation Learning upon Arbitrary Demonstrations by Pre-Training Dynamics Representations
- Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency
- Nonparametric Canonical Correlation Analysis
- Efficient Estimation of Word Representations in Vector Space
- Spectral Bellman Method: Unifying Representation and Exploration in RL
- Reinforcement Learning via Fenchel-Rockafellar Duality
- Representation Learning with Contrastive Predictive Coding
- DINOv2: Learning Robust Visual Features without Supervision
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis