Post-hoc Disentanglement of Textual and Acoustic Features in Pre-trained Speech Encoders
The framework uses an Information Bottleneck principle to separate complex speech representations from pre-trained models into two distinct parts: one encoding textual content and another encoding acoustic features. This method successfully disentangles these entangled features for tasks like emotion recognition, showing that text components predict transcripts well while acoustic components capture relevant non-textual data.