BitLLM: Efficient Low-Bit, Conversion-Hiding LLM Decoding on Mobile GPUs
cs.AR, cs.LG, cs.OS, cs.PF
Submitted: 2026-10-31
Updated: 2026-10-31
Code: https://github.com/NVIDIA/FasterTransformer
Terminology
Sources
- DB-LLM: Accurate Dual-Binarization for Efficient LLMs
- Progressive Mixed-Precision Decoding for Efficient LLM Inference
- Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- Dynamic Stripes: Exploiting the Dynamic Precision Requirements of Activation Values in Neural Networks
- QAQ: Quality Adaptive Quantization for LLM KV Cache
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- The Llama 3 Herd of Models
- Intelligent Virtual Assistants with LLM-based Process Automation
- Transformer-Lite: High-efficiency Deployment of Large Language Models on Mobile Phone GPUs
- Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents
- Graph is a Substrate Across Data Modalities
- QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
- Pointer Sentinel Mixture Models
- Real-time privacy preserving disease diagnosis using ECG signal
- Gemma 2: Improving Open Language Models at a Practical Size
- GPTVoiceTasker: Advancing Multi-step Mobile Task Efficiency Through Dynamic Interface Exploration and Learning
- WhisperFlow: speech foundation models in real time
- Neuralink: Fast LLM Inference on Smartphones with Neuron Co-Activation Linking
- T-MAN: Enabling End-to-End Low-Bit LLM Inference on NPUs via Unified Table Lookup
Related papers
- Embedded Arena: Iterative Optimization via Hardware Feedback
- WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
- SparsePixels: Efficient Convolution for Sparse Data on FPGAs
- Golden Ruler: A Numeric Format Catalog with Bit-Exact Conformance Vectors for FP8, BF16, MXFP4, and Microscaling Formats
- AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
- RFChipAgent: Multi-Agentic AI Flow for Analog/RF Chip Design