PatchKV: Weight-Space Compensation of KV Cache
PatchKV is a training-free method to improve LLM performance when using aggressively compressed KV caches. It compensates for lost context information by adding a context-specific weight patch derived via closed-form ridge regression. This patch aligns block activations between the full and compressed caches, effectively carrying essential context information within the model weights without increasing inference cost.