AbridgeKV: Compressing Long Reasoning into the KV Cache of Learned Pause Tokens
ABRIDGEKV compresses long reasoning chains by summarizing completed text blocks into a few learned 'pause tokens.' These tokens encode the block's information, freeing up the original cache memory. Subsequent tokens then attend to these compact summaries instead of the full block. This significantly reduces memory usage and speeds up inference for long tasks.