Can Automated Feedback Turn Students into Happy Prologians?

Providing personalized feedback is essential for effective learning, but delivering it promptly can be challenging in large-scale courses.In this work, we present PROHELP, an automated assessment platform for Prolog built on top of the GITSEED framework, and we evaluate it through a survey of 144 students from a 365-student undergraduate logic programming course.We assessed the perceived usefulness of seven types of automated feedback, including automatic testing, predicate scoring, syntax error highlighting, open choice point warnings, score rankings, solution type validation, and unknown predicate name suggestions.Our results show that 74% of students agreed the feedback helped increase their grade, and the system achieved a System Usability Scale score of 78.5 (grade B+).Among the feedback types, automatic testing was ranked as the most useful, followed by open choice point warnings and predicate scoring, with statistically significant differences.We found no significant effect of students' interest level, engagement with optional exercises, or use of large language models on their perception of feedback usefulness.We also explore student preferences for future feedback features, finding a significant preference for showing the differences between generated and expected test outputs. Can Automated Feedback Turn Students into Happy Prologians?assessment platform for Prolog built on top of GITSEED.PROHELP includes feedback features ranging from automated testing to typo-correction suggestions and validation of implementation techniques.We designed and deployed a mixed-methods study with the students who interacted with PROHELP.We used survey responses collected during a 9-week undergraduate course with 365 students.Overall, we analyzed 144 survey responses.The main contributions of this work are:• Understanding which types of feedback implemented in PROHELP students find the most helpful;• Gauging students' interest in more complex types of automated feedback they would like to see in the future.Results show that the feedback provided to students was helpful and that the PROHELP automated assessment system is user-friendly.Students also showed great interest in proposed improvements and suggested some themselves.This paper is organized as follows: in Section 2 we present an overview of related work.Following that, in Section 3, we present the course in which this study was conducted, and in Section 4, we describe the automated assessment system and the types of feedback provided.Then, in Section 5, we present the methodology for our study, and in Section 6, we present the analysis and results.Finally, in Section 7, we present a discussion on the obtained results and conclude the paper.