Can Evolution Be Learned as a Generative Process?

The essence of evolutionary optimization may not be random trial-and-error, but rather the implicit learning of the distribution of high-fitness solutions. Based on this insight, we propose viewing evolutionary optimization as a fitness-driven process that explicitly learns and generates high-quality population distributions through a generative model. Inspired by VAEs, we design a family of learnable neural optimizers tailored to different optimization scenarios: the standard VAE is suitable for unimodal unconstrained problems; VQ-VAE and its multi-head extension MH-VQVAE effectively capture multiple optimal modes via discrete latent spaces; ConsVAE embeds the Lagrangian multiplier method into the loss function to accurately locate optimal solutions within the feasible region; and the MOE can uniformly cover complex Pareto fronts. In benchmark evaluations, the proposed methods significantly outperform CMA-ES, OpenES, and NSGA-III on high-dimensional multi-peak and multi-objective optimization tasks (Wilcoxon signedrank test, α = 0.05), and their constrained variants also efficiently handle constrained problems. This work not only provides a set of high-performance novel optimizers, but more importantly advocates a paradigm shift toward "learning to optimize," opening new research directions in the field of black-box optimization. Code is publicly available: https://github.com/Xccc0304/vae_evo.