Understanding When and Where Self-Supervised Speech Models Learn Phonemes and Prosodic Cues through Layer-Wise and Time-Resolved Probing
cs.CL
Submitted: 2026-11-08
Updated: 2026-11-16
Code: https://github.com/facebookresearch/fairseq
Terminology
Sources
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
- HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
- On The Landscape of Spoken Language Models: A Comprehensive Survey
- Self-Supervised Speech Representation Learning: A Review
- Probing self-supervised speech models for phonetic and phonemic information: a case study in aspiration
- Self-Supervised Speech Representations are More Phonetic than Semantic
- Probing for Phonology in Self-Supervised Speech Representations: A Case Study on Accent Perception
- Layer-wise Analysis of a Self-supervised Speech Representation Model
- Comparative layer-wise analysis of self-supervised speech models
- Similarity Analysis of Self-Supervised Speech Representations
- Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representations
- Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models
- Tracking the emergence of linguistic structure in self-supervised models learning from speech
- A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech models
- Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0
- fairseq: A Fast, Extensible Toolkit for Sequence Modeling
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data