SPIN: Shadow Predictive Indexer for Sparse Attention

Name: SPIN: Shadow Predictive Indexer for Sparse Attention
Uploaded: 2026-10-16T00:23:11.871417+00:00
Duration: 815 s
Description: Spin reduces indexer overhead in large language models by using lightweight, history-based prediction instead of scoring every KV block at every step. It predicts important token blocks using exponential moving averages of past scores, skipping up to 40% of the input while maintaining high task quality and improving serving speed.

Spin reduces indexer overhead in large language models by using lightweight, history-based prediction instead of scoring every KV block at every step. It predicts important token blocks using exponential moving averages of past scores, skipping up to 40% of the input while maintaining high task quality and improving serving speed.

Open the full episode