Local minima of the empirical risk in high dimension: beyond convexity

We consider empirical risk minimization whereby the data are Gaussian vectors, the model is parametrized by d x k matrices, and the loss depends on the data via their projection onto the parameter. We use the Kac-Rice formula to derive a bound on the expected number of local minima of this empirical risk in the proportional asymptotics where n, d → ∞, with n ≍ d. Via Markov’s inequality, this bound allows determining the positions of minimizers (with exponential deviation bounds), and hence derive sharp asymptotics on the estimation and prediction error, along other statistics of a minimizer. I will present the general framework and its implications in the convex case, based on [AMS25], and then discuss some of our recent progress for non-convex losses [MS26]. (A = Kiana Asgari, M = Andrea Montanari)