Construct validity of a mixed-worded mathematics confidence scale: Relations with mathematics achievement in TIMSS 2019

This study examines the construct validity of the mathematics confidence scale in the Swedish TIMSS 2019 fourth-grade data, addressing a methodological gap in educational effectiveness research, conducting secondary analyses of international large-scale assessments (ILSAs). Previous research has operationalised the mathematics confidence construct in various ways. Although ISLA data provide robust measures, self-report scales require validation for method effects and linguistic equivalence across cultures. Using confirmatory factor analysis and bifactor exploratory structural equation modelling, this study examined the factorial structure and translation equivalence of the mathematics confidence scale to validate its operationalisation for further analyses. Results indicated method effects associated with mixed-worded items, suggesting that linguistic complexity may compromise validity. Structural equation models confirmed a significant relationship between mathematics confidence and achievement, yet model fit, and strength of relationships varied across operationalisations. The findings underscore the importance of construct validation in ILSA-based secondary analyses and call for greater attention to translation equivalence and item wording.