Teachers’ questions during non-standardised oral exams – construct, assessed domain, and possible implications for validity

Non-standardised oral exams are used in many educational contexts; however, little is known about the differences in how non-standardised oral exams are conducted and how variations in practices affect the exams’ validity. To provide insight into this topic, the current study investigates the questions teachers pose during non-standardised oral exams and discusses how teachers’ questioning may have implications for the exams’ validity. The study is based on 36 video-recorded authentic oral exams in the subject of Norwegian language and literature in four secondary schools. Drawing on content analysis, this study reveals great variations in the questions teachers pose to different students. These variations influence what is assessed and the dependability and interpretation of the exam results; thus, the findings have major implications related to the validity of the exams. Moreover, such variations are not likely to exist only within the Norwegian context but probably affect validity in other educational fields applying non-standardised oral exams as well. As there is an urgent need for further development of methods for educational assessments, due, amongst other things, to students’ use of AI, these findings are important far beyond the Norwegian context.