Application of a Justice-Oriented Content Validation to a Classroom Assessment Measure Literacy Measure

Content validation is critical to instrument development (McCoach, 2013; Sireci & Benitez, 2023). However, recent scholarship suggests that instrument validation efforts may unintentionally contribute to educational inequities and proposes frameworks for justice-oriented test validation (Chang & Cochran-Smith, 2022; Randall et al., 2022). This study is among the first to report on efforts to apply justice-oriented validation to an affective measure. Our scenario-based instrument measures Chilean teacher candidate (TC) assessment literacy. Two content alignment studies assess the instrument's alignment with Chile's Teacher Preparation Standards and its ability to capture a variety of classroom assessment approaches—assessment as, of, and for learning and supremacist, culturally responsive, and culturally sustaining assessment. Results highlight that justice-oriented content validation methods improved the measure. It helped refine conceptions of equitable classroom assessment and modify the instrument to address instances where negative stereotypes about minoritized groups were inadvertently reinforced. As such, the study offers a model for applying justice-oriented content validation frameworks to affective instruments.