From log-likelihood to probability-likelihood: A novel approach to enhancing classification performance by in-depth Bayesian inference

Bayesian Network Classifiers (BNCs) graphically model the probabilistic relationships among variables in the form of a directed acyclic graph (DAG). The log-likelihood function is commonly applied to evaluate the fit of DAG to data, and the significance of dependencies in DAG is measured by information metrics rather than probability metrics. However, the fit to data and the significance of dependencies may not hold from the perspective of probability theory. To address this issue, in this paper we propose to apply probability-likelihood function to provide a comprehensive and fundamental description of the mapping relationship between BNCs and data. Based on this, a heuristic search technique is introduced to construct a maximum probability-likelihood spanning tree by allowing for in-depth Bayesian inference of the probabilistic (in)dependence between attributes. Furthermore, instance-based learning is applied to enhance the interpretability and advance the classification capabilities of the learned BNCs in various domains and applications. Case analysis in the field of water quality monitoring shows that the proposed Average Weighted Bayesian Classifier (AWBC) can not only achieve accurate classification of water quality grades, but also fully capture and express the complex inherent dependence relationships among various water quality variables. Experimental evaluations conducted on 30 benchmark datasets from the University of California, Irvine (UCI) Machine Learning Repository demonstrate that, the final BNCs deliver superior or comparable classification performance compared to other BNC learners.