KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models
The episode discusses KVDiagnosis, a benchmark for diagnosing why KV-cache compression fails in long-context language models. Hosts explain that aggregate scores hide failure causes, and KVDiagnosis uses diagnostics like evidence coverage and likelihood drift to classify failures. They highlight that 63.2% of failures stem from evidence loss, and a targeted intervention repairs 29.2% of low-EAR failures.