Empirical analysis of authenticity in reality-based mathematical test items

For assessing students’ ability to use mathematics in real-world contexts, reality-based mathematical test items are particularly important. In these items, authenticity is a key quality criterion, indicating whether real-world references in test items are genuine. Although widely used in large-scale studies, authenticity in test items has rarely been empirically analysed. The study aims to improve understanding of this concept by investigating the added value of the Model for Authenticity in Modelling Test Items (MAMTI) for assessing authenticity in reality-based test items. Using 109 items from the German large-scale test VERA 8, the study coded authenticity aspect-wise, analysed the resulting distributions, examined correlations with general item characteristics, and conducted a hierarchical cluster analysis. The results show that the occurrence of authenticity varies greatly across different test item aspects, aspects of authenticity can be described in relation to general item characteristics, and different “clusters of items” can be identified based on aspects of authenticity.