Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression
The episode discusses 'Last But Not Least,' a method for improving multimodal KV cache compression. It addresses the weakness of simply averaging attention, which can miss sparse evidence. The solution calibrates standard scoring using boundary evidence from the last query, achieving significant efficiency gains while maintaining accuracy across various visual tasks.