High-dimensional networks and mean squared error for possibly misspecified models

To avoid missing important variables and their connections in networks, more andmore variables are included in network analysis. Here we show that in a settingwith many more parameters than observations (high-dimensional) it is possible toget an accurate estimate of the right model (which connections are in the network).However, there are two interesting cases: (i) If the model specified for estimation isapproximately correct, then model selection methods work reasonably well, and (ii)if the model is misspecified, then model selection procedures require a penalty forthe high dimensions, represented by a correct specification of the model space. Herewe show the impact of the ridge parameter on the mean squared error, and how thisleads to low test variance and hence to models with large numbers of parameters thatgeneralise well to other samples. We connect these insights with results from machinelearning, where the so-called double descent (when more parameters are includedthan observations, the mean squared error goes down a second time) has put thetraditional view on model selection upside down. Essentially, for adequate modelselection in models with a large number of parameters, the volume of the modelspace needs to be included in the penalty. However, if the model is misspecified andthe penalty is based on the misspecified model, then it is possible that a model withmany more parameters than are in the true underlying model will be selected; theinterpretation of parameters is then dicult but makes machine learning successful.