HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization
The episode discusses HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization, a method by Artur Zagitov and Gleb Molodtsov. Hosts explore how HARP fundamentally rethinks data preprocessing for LLMs, using a learnable orthogonal processor to achieve high efficiency and quality gains during quantization.