Studying the Soupability of Documents in State Space Models

Name: Studying the Soupability of Documents in State Space Models
Uploaded: 2026-10-06T10:24:19.250782+00:00
Duration: 1113 s
Description: Researchers investigated 'soupability,' a method to merge hidden states from independently encoded documents in State Space Models (SSMs). By pooling these per-document representations using averaging, they showed that finetuned Mamba2 models perform competitively across multi-hop QA and long-document reasoning. This technique allows for scalable, modular reasoning over large document sets.

Researchers investigated 'soupability,' a method to merge hidden states from independently encoded documents in State Space Models (SSMs). By pooling these per-document representations using averaging, they showed that finetuned Mamba2 models perform competitively across multi-hop QA and long-document reasoning. This technique allows for scalable, modular reasoning over large document sets.

Open the full episode