Multivariate Bayesian inversion for classification and regression

Abstract We propose the statistical modeling approach to supervised learning (i.e., predicting labels from features) as an alternative to algorithmic machine learning (ML). The approach is demonstrated by employing a multivariate general linear model (MGLM) describing the effects of labels on features, possibly accounting for covariates of no interest, in combination with prior distributions on the model parameters. ML "training" is translated into estimating the MGLM parameters via Bayesian inference and ML "testing" or application is translated into Bayesian model comparison—a reciprocal relationship we refer to as multivariate Bayesian inversion (MBI). We devise MBI algorithms for the standard cases of supervised learning, discrete classification and continuous regression, derive novel classification rules and regression predictions, and use practical examples (simulated and real data) to illustrate benefits of the statistical modeling approach: interpretability, incorporation of prior knowledge, probabilistic predictions. We close by discussing further advantages, disadvantages, and the future potential of MBI.