DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving

Name: DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving
Uploaded: 2026-10-19T02:35:00.265809+00:00
Duration: 206 s
Description: Serving Mixture-of-Experts (MoE) models involves two costs: storing every expert and decoding only the selected ones. DuoRate solves this by storing weights once and reading them per workload using a Lagrangian rate-distortion allocation. This method achieves superior accuracy compared to existing 2-bit baselines, matching 4-bit formats under long decoding conditions.

Serving Mixture-of-Experts (MoE) models involves two costs: storing every expert and decoding only the selected ones. DuoRate solves this by storing weights once and reading them per workload using a Lagrangian rate-distortion allocation. This method achieves superior accuracy compared to existing 2-bit baselines, matching 4-bit formats under long decoding conditions.

Open the full episode