ActQuant: Sub-4-bit Action-Guided Backbone Weight-Only Quantization for Vision-Language-Action Models
cs.CV, cs.AI
Submitted: 2026-05-19
Updated: 2026-11-05
Code: https://github.com/ggml-org/ggml
Project page: https://actquant.github.io
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Vision-Language-Action (VLA) models exhibit remarkable action generation for embodied intelligence, but their heavy compute make deployment on edge platforms impractical.
Terminology
Abstract
Vision-Language-Action (VLA) models exhibit remarkable action generation for embodied intelligence, but their heavy compute make deployment on edge platforms impractical. Aggressive, sub-4-bit weight quantization is the natural solution, yet existing post-training quantization (PTQ) methods suffer severe performance degradation in this regime. To address this, we introduce ActQuant, an action-guided mixed-precision PTQ framework that operates in two stages: (1) an inter-tensor bit allocator that assigns each weight matrix a single bit-width based on how much it contributes to predicting the agent's actions; (2) an intra-tensor scale optimizer tunes per-block quantization scales using action-aware curvature, so that dynamic range is concentrated on the weights most influential for control. To deliver the on-device benefits of our aggressive quantization, we further introduce OmniModel.cpp, an agentic conversion pipeline that ports architectures into a native C/C++ runtime with efficient low-bit kernels. We evaluate ActQuant both in simulation and on a real-world 6-DoF UR3 arm, with all models deployed through OmniModel.cpp. On the LIBERO benchmark, ActQuant is the only method that operates at or below 3 bits-per-weight, retaining 95.0% on OpenVLA-OFT and 94.8% on π 0.5. Pushed further, ActQuant reaches 2.5 bpw at 90.1% on OpenVLA-OFT, compressing the backbone from 14.3 GB to 2.7 GB (5.3 times). On the physical UR3 arm, π 0.5 quantized with ActQuant retains the baseline's success rate while reducing the memory footprint by 2.5 times.
Sources
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- Channel-Wise Mixed-Precision Quantization for Large Language Models
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
- Extreme Compression of Large Language Models via Additive Quantization
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
- OpenVLA: An Open-Source Vision-Language-Action Model
- SqueezeLLM: Dense-and-Sparse Quantization
- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
- H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
- A Survey on Vision-Language-Action Models: An Action Tokenization Perspective
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs