Seeing Different Stories: Human Variability in Statistical Assumption Checking and Model Selection

View at publisher Summarise with VIRAT
Authors: Gavin Tovar, Sarah C. Emerson
Cited by: 0
References: 19

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Inspired by recommendations commonly encountered in introductory statistics classes to perform model checking and make modeling decisions based on visual inspections of diagnostic plots, we implemented and administered a survey on interpretation of a variety of diagnostic plots across multiple settings. We summarize the patterns in the answers provided by more than 300 undergraduate and graduate students enrolled in statistic courses via a mixed effects model exploring the impact of different data-generating settings and various plot features. Notably, we find that the students surveyed here are comparable to random chance at making the correct decisions on the basis of the plots they were shown. The implications of this uncertainty and the lack of reproducibility introduced by the model checking/model selection process are explored in depth in one of the settings presented to demonstrate the consequences of using subjective and frequently sub-optimal human judgments to make modeling choices.

Fields of Study & Subjects

Statistics Education and Methodologies Statistical Methods and Bayesian Inference Statistical Methods and Applications

Publication details

Journal / source
The American Statistician
Volume
80
Issue
4
Pages
591–608
Publication date
26 March 2026
Language
EN