Voice-Native LLMs Hear Frustration but Do Not Listen to It
cs.SD, cs.AI, eess.AS
Submitted: 2026-11-02
Updated: 2026-11-02
Code: https://github.com/Astarojth/hear-not-listen
Terminology
Sources
- Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMA
- When Models Hear What They Expect: Diagnosing Prosodic Heuristics in Multimodal Sarcasm Detection
- Evaluating Emotion Recognition in Spoken Language Models on Emotionally Incongruent Speech
- Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)
- Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variations
- Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models
- Large Language Models Understand and Can be Enhanced by Emotional Stimuli
- emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation
- Do Audio LLMs Listen or Read? Analyzing and Mitigating Paralinguistic Failures with VoxParadox
- Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysis
- CLASH: Counterfactual Auditing of Lexical and Prosodic Reliance in Spoken Sarcasm Detection
- EmoOmni: Bridging Emotional Understanding and Expression in Omni-Modal LLMs
- A Looming Replication Crisis in Evaluating Behavior in Language Models? Evidence and Solutions
- HumDial-EIBench: A Human-Recorded Multi-Turn Emotional Intelligence Benchmark for Audio Language Models
- MiMo-Audio: Audio Language Models are Few-Shot Learners
- Qwen2.5-Omni Technical Report
- Qwen3-Omni Technical Report
- Resurfacing Paralinguistic Awareness in Large Audio Language Models
- Instruction-Following Evaluation for Large Language Models
Related papers
- Speech-DRAME: A Framework for Human-Aligned Benchmarks in Speech Role-Play
- MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control
- Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems