Hearing Like Humans? Sound Symbolism and Perceptual Alignment in Speech Language Models
Researchers tested if Speech Language Models (SLMs) intuitively map speech sounds to visual shapes like roundness or pointedness, mirroring human sound symbolism. The study found SLMs struggle with auditory judgments because they miss crucial acoustic cues humans use, but they successfully match shape perceptions when given visual input. This suggests the weakness is in how speech is represented audibly.