Activation Sensitivity as a Unifying Principle for Post-Training Quantization
This work proposes a unified theory for post-training quantization by defining 'activation sensitivity' as the expected impact of channel perturbations on loss. It formalizes this using gradient-weighted activations to create a metric that captures both channel usage and error propagation. This framework shows how existing methods like AWQ and GPTQ are approximations of this fundamental quantity, highlighting where they fall short.