CONSTRUCTIVE BY DESIGN? EVALUATING AN AI FEEDBACK COACH IN COLLABORATIVE LEARNING CONTEXTS
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Given that LLMs (large language models) can produce fluent, scalable, and context-sensitive responses on request, often simulating human conversation, many higher education institutions are considering their use for feedback purposes. It is tempting to consider LLMs as suitable replacements or supplements for limited teacher feedback in large student cohorts. However, LLM-based educational technologies are being introduced faster than their consequences can be fully characterized; effects on student-agent interactions, feedback quality, and learning outcomes remain insufficiently understood. As such, we investigated an LLM-based AFC (Automated Feedback Coach) that provides feedback-on-feedback while students compose peer-review comments in an undergraduate academic skills course. In a mixed-methods case study, we compared an intervention cohort using the AFC with a control cohort without it. Students’ and lecturers’ perceptions and reported feedback behavior were examined through a student survey and focus groups, supplemented by lecturer notes. To assess effects on student work, peer feedback comments were analyzed using quantitative language and sentiment metrics, along with qualitative coding of clarity, constructiveness, and tone. Results show that students largely accepted the AFC, describing it as useful and trustworthy and reporting gains in confidence and perceived feedback quality. In contrast, analyses of peer-feedback comments showed only minor differences: the AFC cohort produced slightly more complex feedback, without meaningful improvements in clarity, constructiveness, or tone. Nevertheless, qualitative evidence indicated negative effects on the learning process, as the AFC seemed to elicit assurance-seeking and tool-satisficing behavior rather than critical reflection. Moreover, students and lecturers perceived the AFC as distracting during the peer feedback writing process and reported variable AFC feedback quality, including generic or inconsistent guidance and sensitivity to differences in phrasing. Based on our findings, we recommend a cautious, evidence-informed approach to developing and integrating LLM-based agents into formative assessment strategies, rather than adopting them solely on the basis of prevailing trends or student evaluations.
Fields of Study & Subjects
Student Assessment and Feedback
Artificial Intelligence in Education
Intelligent Tutoring Systems and Adaptive Learning
Publication details
- Journal / source
- INTED proceedings
- Volume
- 1
- Publication date
- 1 March 2026
- Language
- EN