Error estimates between SGD with momentum and underdamped Langevin diffusion
Abstract
Stochastic gradient descent with momentum is a popular variant of stochastic gradient descent, which has recently been reported to have a close relationship with the underdamped Langevin diffusion. In this paper, we establish time-uniform error bounds between them in the 1-Wasserstein and total variation distances. Leveraging Malliavin analysis and the exponential ergodicity of the underdamped Langevin diffusion, we characterize the long-time behavior of this algorithm, even for nonconvex objective functions.
Fields of Study & Subjects
Advancements in Semiconductor Devices and Circuit Design
Atomic and Subatomic Physics Research
stochastic dynamics and bifurcation
Publication details
- Journal / source
- The Annals of Applied Probability
- Volume
- 36
- Issue
- 4
- Pages
- 3295–3344
- Publication date
- 1 August 2026
- Language
- EN