The Trust Process Applied to Machine Learning Algorithms: The Influence of Calibrated Confidence Estimates

Little research has been conducted on user interactions with machine learning models and users' trust. The current study explores differences in psychological perceptions of trust between two machine learning models. Across three studies, participants (N= 130, 134, 132) were recruited online and completed an image-binning monitoring task with either an uncalibrated classification model (UCM) or a calibrated classification model (CCM) that provided confidence estimates about their decisions. The UCM was highly confident, regardless of images from out-of-distribution classes, whereas the CCM was better calibrated between accuracy and confidence of out-of-distribution images. Results revealed participants performed better on the task in the CCM condition across all studies. In addition, trustworthiness perception results differed depending on whether the study included out-of-distribution images. Overall, calibrated confidence estimates displayed by CCMs can be an effective means of increasing transparency and enhancing our understanding of trust in machines, but more research is needed on trust perceptions in image classification tasks.