A Comprehensive Evaluation on Quantization Techniques for Large Language Models
This study thoroughly reviews and experimentally evaluates post-training quantization methods for large language models. By separating pre-quantization transformations from error mitigation, researchers found that optimizing rotation and scaling yields the best results. They also discovered that asymmetric quantization benefits activations more than weights, and the format of scaling factors significantly impacts performance across different FP4 data types.