Anchored Stein Variational Gradient Descent Algorithm for Non-smooth Sampling

We introduce a Stein variational framework for sampling from a target density p proportional to e-U when the target potential U is possibly non-smooth. The key idea is to introduce a smoother reference potential U 0 and the anchor weight a=e U-U 0. The resulting anchored Stein variational gradient descent (SVGD) algorithm is able to target p proportional to e-U without computing the gradient of U and include standard SVGD as a special case. In particular, we allow the target potential U to be locally Lipschitz while the reference potential U 0 is continuously differentiable, along with other suitable assumptions. Under this low regularity regime, we establish a Kullback-Leibler first variation formula for anchored perturbations, identify the corresponding steepest descent direction in the reproducing kernel Hilbert space and prove a nonasymptotic convergence bound for the anchored SVGD algorithm in terms of an anchored kernelized Stein discrepancy. Finally, we also provide guidance on constructing a suitable reference potential U 0 based on U.