GRAU: A Generic Runtime-Reconfigurable Activation-Requantization Unit for Quantized Neural Network Accelerators
The Generic Reconfigurable Activation Unit (GRAU) is a novel hardware design for neural network accelerators that uses piecewise linear fitting with power-of-two approximations. It overcomes limitations of traditional multi-threshold units by achieving high efficiency and flexibility. GRAU supports mixed precision, non-monotonic activations like SiLU, and allows runtime reconfiguration using only basic comparators and shifters, resulting in over 90% reduction in LUT consumption.