Generative machine learning approaches to optimization

We present a generative machine learning approach to optimization problems. The key idea is that generative models can learn the joint probability distribution of variables and their derivatives, or transform simple distributions to target distributions. By conditioning on desired properties (e.g., derivatives equal to zero), solutions are obtained by sampling. We demonstrate this using Gaussian mixture models, which approximate joint distributions, and conditional flow matching, which learns distribution transformations. Examples include root finding, unconstrained and constrained optimization, parameter estimation, and state space mapping. This pedagogical work shows that generative models offer a fundamentally different approach to optimization, one that is data-driven, does not require iterative solvers, and naturally handles problems with multiple solutions by representing the full solution distribution. We discuss limitations and conclude the approach shows promise as an alternative to conventional methods.