Finite-dimensional Gaussian approximation for deep neural networks: Universality in random weights
Abstract
We study the Finite-Dimensional Distributions (FDDs) of deep neural networks with randomly initialized weights that have finite-order moments. Specifically, we establish Gaussian approximation bounds in the Wasserstein-1 norm between the FDDs and their Gaussian limit assuming a Lipschitz activation function and allowing the layer widths to grow to infinity at arbitrary relative rates. In the special case where all widths are proportional to a common scale parameter n and there are L−1 hidden layers, we obtain convergence rates of order n−(1∕6)L−1+ϵ, for any ϵ>0.
Fields of Study & Subjects
Neural Networks and Applications
Gaussian Processes and Bayesian Inference
Tensor decomposition and applications
Publication details
- Journal / source
- Bernoulli
- Volume
- 32
- Issue
- 4
- Pages
- 3042–3062
- Publication date
- 31 July 2026
- Language
- EN