Variational Bayes neural networks for high-dimensional non parametric regression: Minimax optimality and adaptivity

.In this article, we study (ultra) high-dimensional non parametric regression problems, where the number of predictors is allowed to (even exponentially) diverge as the sample size grows. We first introduce a new class of high-dimensional non parametric regression models that assume the true regression function consists of several component functions, each of which lies on a subspace spanned by a small number of predictors. This function class includes a standard sparse regression model, sparse additive and interaction models, and single- and multi-index models as special cases. To perform optimal inference for the proposed model, we develop a variational deep learning method with a continuous spike-and-slab prior, which offers efficient computation. We prove that the proposed variational neural network attains a minimax optimal contraction rate. Notably, the contraction rate is simultaneously adaptive to both the unknown smoothness and sparsity of the true regression function. We examine the effectiveness of our method in a simulation study.