Mean field optimization problem regularized by Fisher information
Abstract
Recently there is a rising interest in the research of mean field optimization, in particular because of its role in analyzing the training of neural networks. In this paper by adding the Fisher information as the regularizer, we relate the regularized mean field optimization problem to a so-called mean field Schrödinger (MFS for short) dynamics. We develop an energy-dissipation method to show that the marginal distributions of the MFS dynamics converge exponentially quickly towards the unique minimizer of the regularized optimization problem. Remarkably, the MFS dynamics is proved to be a gradient flow on the probability measure space with respect to the relative entropy. Finally, we propose a Monte Carlo method to sample the marginal distributions of the MFS dynamics.
Fields of Study & Subjects
Stochastic Gradient Optimization Techniques
Neural Networks and Applications
Quantum many-body systems
Publication details
- Journal / source
- The Annals of Applied Probability
- Publisher
- Institute of Mathematical Statistics
- Volume
- 36
- Issue
- 2
- Pages
- 1002-1047
- Publication date
- 1 April 2026