Random Feature Gaussian Process Attention: Linear-Time Probabilistic Attention with Calibrated Uncertainty
This work addresses poor uncertainty calibration in Transformers by modeling attention as a Gaussian Process (GP) posterior. The proposed Random Feature Gaussian Process Attention (RFF-GPA) module approximates this GP using random Fourier features, achieving linear time complexity for mean and variance estimation. This allows for principled uncertainty calibration while significantly reducing computational cost compared to previous methods.