WhiteMatter: All-to-All Cross-Layer Connections via KV Source Mixing

Name: WhiteMatter: All-to-All Cross-Layer Connections via KV Source Mixing
Uploaded: 2026-10-03T23:59:23.199565+00:00
Duration: 920 s
Description: WhiteMatter connects every attention layer to representations from all layers of past tokens by dynamically mixing hidden states. This allows layers to utilize deeper representations during decoding, improving performance over standard models while maintaining competitive computational costs.

WhiteMatter connects every attention layer to representations from all layers of past tokens by dynamically mixing hidden states. This allows layers to utilize deeper representations during decoding, improving performance over standard models while maintaining competitive computational costs.

Open the full episode