Bayesian-Optimized Ensemble Learning for Music Popularity Prediction with Shapley-Based Interpretability

Music popularity prediction is a fundamental problem in music information retrieval, with important implications for digital content dissemination and creative decision-making on streaming platforms. In this study, music popularity prediction is formulated as a supervised regression problem, and six widely-used tree ensemble models (Random Forest, XGBoost, CatBoost, LightGBM, Extra Trees, and Decision Tree) are systematically evaluated using large-scale Spotify data. Among these models, Random Forest achieves the best predictive performance on this dataset (RMSE = 6.79, MAE = 5.10, and R2 = 0.6658), followed by Extra Trees (R2 = 0.6378) and Decision Tree (R2 = 0.6328). Bayesian hyperparameter optimization based on a Tree-structured Parzen Estimator with an Expected Improvement acquisition function is conducted over 50 trials with 5-fold cross-validation to ensure robust model selection. Shapley value decomposition via SHAP analysis reveals that temporal recency dominates feature importance, far surpassing traditional musical attributes, while acoustic intensity (loudness) exhibits a U-shaped contribution pattern with optimal values at moderate intensity levels. Further SHAP dependence analysis uncovers non-linear relationships, indicating substantial popularity advantages for recent releases and optimal loudness levels around −5 to 0 dB. These findings suggest that streaming popularity is primarily governed by temporal exposure dynamics and production-related characteristics rather than intrinsic musical structure, offering both theoretical insights for music information retrieval research and suggestive empirical patterns that may inform future investigations into digital music ecosystems.