AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving

arXiv:2604.26103 · cs.AR, cs.AI, cs.DC, cs.LG · Submitted 2026-04-28 · Read on arXiv

cs.AR, cs.AI, cs.DC, cs.LG

Submitted: 2026-04-28

Updated: 2026-10-30

Terminology

• Decode — 2× in this paper · explained in 2 episodes
• Communication Overhead — 1× in this paper · explained in 2 episodes

Related papers

• WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
• SparsePixels: Efficient Convolution for Sparse Data on FPGAs
• Golden Ruler: A Numeric Format Catalog with Bit-Exact Conformance Vectors for FP8, BF16, MXFP4, and Microscaling Formats
• RFChipAgent: Multi-Agentic AI Flow for Analog/RF Chip Design
• PoisonCap: Efficient Hierarchical Temporal Safety for CHERI
• Provisioning to Runtime Optimization of a 100 MW-Scale AI Cluster