Trust Region Q Adjoint Matching
Trust Region Q-Adjoint Matching (TRQAM) is a stable method for fine-tuning pretrained flow policies in off-policy reinforcement learning. It addresses instability by controlling path-space KL divergence using a trust-region parameter lambda inside the sampling dynamics. This allows it to adaptively balance exploiting the learned critic against staying close to the original policy, achieving high success rates on various benchmarks.