Quantifying AI Model Trust as a Model Sureness Measure by Bidirectional Active Processing and Visual Knowledge Discovery

Trust in machine learning models is critical for deployment by users, especially for high-risk tasks such as healthcare. Model trust involves more than just performance metrics such as accuracy, precision, or recall, it includes user readiness to let a model make decisions. Trust is commonly associated with model prediction stability under variations to training data, noise, parameters, explanations, etc. This paper expands on former model trust concepts with a proposed Model Sureness measure. Model Sureness in this work quantifies stability of model accuracy under variations to the training data for any model by a bidirectional active learning with Visual Knowledge Discovery method. This method iteratively retrains a model on varied training data until a user-defined criterion is met, e.g., 95% test data accuracy. This finds a smaller sufficient training data set for a model to meet the criterion. Then Model Sureness is the ratio of the number of unnecessary cases to all cases in training data. The grater ratio indicates high model sureness in accordance with this measure. Conducted case studies on three common benchmark datasets from biology, medicine, and handwriting recognition show well-preserved model accuracy and high sureness of the respective models. Specifically, removal of unnecessary cases was from 20% to 80% and on average about 50% of training data.