FlyWheel: Attention Kernels Built for the TPU
cs.CL, cs.LG
Submitted: 2026-11-01
Updated: 2026-11-01
Code: https://github.com/ax-ml/jax
Terminology
Sources
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Ragged Paged Attention: A High-Performance and Flexible LLM Inference Kernel for TPU
- ReplaySSM: Cache SSM Inputs, Not State
- Online normalizer calculation for softmax
- Fast and Stable Triangular Inversion for Delta-Rule Linear Transformers
- VFA: Relieving Vector Operations in Flash Attention with Global Maximum Pre-computation
- Kimi Linear: An Expressive, Efficient Attention Architecture
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data