Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs

Name: Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs
Uploaded: 2026-10-06T10:56:23.708671+00:00
Duration: 296 s
Description: This work developed a framework to compress large vision-language models like Llama 3.2 11B Vision Instruct down to 3.7 GB using a novel 2.7-bit format called S3D8. They used synthetic data generation and quantization-aware training (QAT) to achieve this compression while maintaining strong performance on visual question answering tasks, making it viable for mobile devices.

This work developed a framework to compress large vision-language models like Llama 3.2 11B Vision Instruct down to 3.7 GB using a novel 2.7-bit format called S3D8. They used synthetic data generation and quantization-aware training (QAT) to achieve this compression while maintaining strong performance on visual question answering tasks, making it viable for mobile devices.

Open the full episode