Homomorphic Advantage Operator: Stabilizing Reinforcement Learning Under Fully Homomorphic Encryption Constraints
The Homomorphic Advantage Operator (HAO) framework stabilizes reinforcement learning under Fully Homomorphic Encryption by preventing 'Bellman drift.' This drift occurs when sequential value estimation errors accumulate, pushing network outputs outside safe polynomial approximation bounds. HAO uses a specific projection matrix to directly center temporal-difference targets, eliminating this error without needing extra non-linear operations.