Using multimodal learning analytics to study student engagement and teacher noticing in mathematics classroom: two different human-in-the-loop workflows

Abstract “Human-in-the-loop” has been the recommended approach to incorporate AI in education research to mitigate risks associated with automation. However, currently there are no specific models or guidelines for designing or reporting on the human-in-the-loop process in education research. This theoretical paper contrasts two different ways in which human judgement and agency were incorporated in the design of two studies in mathematics education. The two studies, one on student collaborative problem solving, and one on teacher noticing and in-the-moment decision making, both used multimodal learning analytics incorporating machine learning to analyse the audiovisual data but involved human input in different ways. Through explicating the research purposes and foci of the two studies, this paper argues that how and when human input should be incorporated in the research process would depend on the research questions and the constructs of interest. By making visible decision points in the research process that incorporates human input in two concrete research examples, this paper offers researchers a reference point for designing future research that incorporates AI within mathematics education and beyond.