SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization
SERQ introduces a method for end-to-end 4-bit quantization of weights and activations using a single low-rank matrix. It achieves high accuracy by jointly mitigating errors from both activation outliers and salient weight rows. The approach uses static activation flattening, saliency-aware error reconstruction via a low-rank compensator, and offline weight permutation to minimize inference latency.