Bayesian Cluster Weighted Gaussian Models

Abstract We introduce a new framework for Bayesian inference in mixtures of normal linear regression models with random covariates. Such types of mixtures belong to the category of cluster-weighted models. The proposed Bayesian cluster-weighted model aims to encompass potential heterogeneity in the distribution of the response variable as well as in the multivariate distribution of the covariates for detecting signals relevant to the underlying latent structure. Of particular interest are potential signals originating from: (i) the linear predictor structures of the regression models and (ii) the covariance structures of the covariates. We model these two components using a lasso shrinkage prior for the regression coefficients and a graphical lasso shrinkage prior for the covariance matrices. A fully Bayesian approach is followed for estimating the number of clusters by treating the number of mixture components as random and implementing a trans-dimensional telescoping sampler. Alternative Bayesian approaches based on overfitting mixture models or using information criteria to select the number of components are also considered. The proposed methodology is compared to mixtures of regressions, mixtures of experts, and existing cluster-weighted models in simulation studies and an application to a biomedical dataset.