On the Rate of Convergence of an Over-Parametrized Deep Neural Network Regression Estimate Learned by Gradient Descent
Abstract
Nonparametric regression with random design is considered. The $L_2$ error with integration with respect to the design measure is used as the error criterion. An over-parametrized deep neural network regression estimate with logistic activation function is defined, where all weights are learned by gradient descent. It is shown that the estimate achieves a nearly optimal rate of convergence in case that the regression function is $(p,C)$--smooth.
Fields of Study & Subjects
Stochastic Gradient Optimization Techniques
Neural Networks and Applications
Machine Learning and Algorithms
Publication details
- Journal / source
- IEEE Transactions on Information Theory
- Volume
- 72
- Issue
- 3
- Pages
- 1777–1797
- Publication date
- 14 January 2026
- Language
- EN