Diffusion at Absolute Zero: Langevin Sampling Using Successive Moreau Envelopes

Abstract. We propose a method for sampling from Gibbs distributions of the form [Formula: see text] that leverages a family [Formula: see text] of approximations of the target density which is deliberately constructed such that [Formula: see text] exhibits favorable properties for sampling when [Formula: see text] is large and such that [Formula: see text] approaches [Formula: see text] as [Formula: see text] approaches 0. This sequence is obtained by replacing (parts of) the potential [Formula: see text] with its Moreau envelope. Through the sequential sampling from [Formula: see text] for decreasing values of [Formula: see text] by a Langevin algorithm with appropriate step size, the samples are guided from a simple starting density to the more complex target quickly. We prove that [Formula: see text] is Lipschitz continuous in the total variation distance and Hölder continuous in the Wasserstein-[Formula: see text] distance, that the sampling algorithm is ergodic, and that it converges to the target density without assuming convexity or differentiability of the potential [Formula: see text]. In addition to the theoretical analysis, we show experimental results that support the superiority of the method in terms of convergence speed and mode-coverage of multimodal densities over current algorithms. The experiments range from one-dimensional toy-problems to high-dimensional inverse imaging problems with learned potentials.