KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
KVEraser is a learned method to efficiently erase a span of text from an LLM's KV cache without recomputing everything. It works by training an eraser module that predicts specific 'steering states' to replace the erased tokens. This allows the system to reuse the rest of the cache, achieving near-perfect erasure performance across long contexts with only a small latency increase.