CompactAttention: Accelerating Chunked Prefill with Block-Union KV Selection
CompactAttention proposes a new chunked prefill attention mechanism using Block-Union KV Selection. Instead of traditional sparse kernels, it treats 2D block-sparse masks as selection signals to build GQA-aware KV block tables. This allows selected Key/Value blocks to be accessed in place during paged execution without needing explicit KV compaction, achieving significant speedups while maintaining high accuracy.