PHBA: Prefix-State Hybrid Block Attention
The Prefix-State Hybrid Block Attention (PHBA) balances long-context modeling and precise retrieval by combining top-k block-sparse retrieval with compact prefix states. It retrieves relevant historical blocks at full resolution while using recurrent prefix states to summarize preceding context, allowing the model to access both specific evidence and broader history efficiently.