Imbalance-Aware Bayesian Multinomial Logistic Regression Via Adaptive Entropy-Based Observation Weighting

Read full text Summarise with VIRAT
Authors: Funmilola Mary Taiwo, Cüneyt Gürcan Akçora, Saman Muthukumarana
Cited by: 0
References: 18

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Class imbalance poses a fundamental challenge in multi-class classification, where standard maximum-likelihood methods systematically favor the majority classes. We develop an Imbalance-Aware Bayesian Multinomial Logistic Regression (IA-BMLR) model that adapts a weighted loss framework, originally developed for variational autoencoders, to Bayesian discriminative classification. The weighting approach combines inverse-frequency class weights with entropy-based weights derived from a Local Entropy Score (LES) that emphasizes observations in ambiguous, high-entropy regions. We treat the entropy weight parameter as a learnable variable with a half-normal prior, rather than a fixed parameter, thereby enabling data-driven calibration via Bayesian inference. We evaluate IA-BMLR on nine benchmark datasets that span severe to mild class imbalance. On the Yeast (ten classes, imbalance ratio = 92.6) and the Lymphography (four classes, imbalance ratio = 40.5) datasets, IA-BMLR achieves a G-Mean of 0.302 and 0.456, respectively. In contrast, all baseline methods, including XGBoost, Random Forest, SVC, Multinomial logistic regression, and unweighted Bayesian Multinomial logistic regression, achieve an exact zero, indicating prediction failure for at least one minority class. IA-BMLR achieves the highest G-Mean across three of nine datasets, demonstrating its reliability in minority-class prediction while maintaining competitive performance on standard metrics.

Fields of Study & Subjects

Imbalanced Data Classification Techniques Machine Learning and Data Classification Data Stream Mining Techniques

Publication details

Journal / source
Sri Lankan Journal of Applied Statistics
Volume
27
Issue
1
Pages
181–203
Publication date
31 March 2026
Language
EN