HOTS-based science questions as assessment instrument for elementary school students: Examining validity and reliability

Assessments based on Higher-order Thinking Skills (HOTS) are critical for elementary students as they are important; however, standardized tools are still lacking. Hence, this study aims to develop valid and reliable HOTS-based science questions as assessment instruments to measure elementary school students’ mastery of science subjects and HOTS. This study employed the Design and Development Research (DDR) model to create HOTS-based science questions for grades four, five, and six at the elementary level. The research involved analysis, design, development, and evaluation phases to produce six summative science questions for each grade. Qualitative data were obtained through focus group discussions (FGD) to align with curriculum analysis. Quantitative data were collected via validation sheets and trial tests to assess validity and reliability. The results indicate that there are two science topics with three HOTS-based instructional objectives for each grade. These instructional objectives were developed using three question types: multiple-choice questions (MCQs) with four options, complex MCQs, and essay questions. MCQs were used to evaluate students’ analytical skills, complex MCQs to assess evaluative skills, and essay questions to measure creative skills and metacognitive knowledge. The developed HOTS-based science questions were deemed valid (CVR = +0.73, CVI = +1.00) and reliable (ICC = 0.914, Cronbach’s α > 0.80). This study suggests that the developed HOTS-based science questions can be utilized by elementary science teachers to assess students’ mastery of science subjects and HOTS.