Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Q-PACE proposes a new method for adaptive mixed-precision training by using a second-order sensitivity model. It estimates layer curvature to predict loss increases from quantization noise, allowing it to dynamically assign bit-widths per layer during training. This approach consistently outperforms existing methods, achieving lower memory usage while maintaining or improving final model accuracy.