Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
cs.CL
Submitted: 2025-10-21
Updated: 2026-11-06
Code: https://github.com/FibonaccciYan/Adamas
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
- MoBA: Mixture of Block Attention for Long-Context LLMs
- Compressive Transformers for Long-Range Sequence Modelling
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Efficient Streaming Language Models with Attention Sinks
- FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data