PostCommit: GenAI Powered Code Reflections for Computer Science Students

Programmatic assignments compose large portions of a student's responsibility in many Computer Science (CS) curricula, aiming to cement and assess Code-to-Concept (C2C) mapping. However, in the era of Generative AI (GenAI), students in CS courses may complete many assignments without mastering any C2C proficiency, leaving valuable knowledge units behind that will be demanded in subsequent, more complex work. Enter PostCommit: a tool for instructors to conduct individualized technical interview style code questions asking students to explain (in plain-language) portions of their submitted work. By providing a second round of assessment for C2C proficiency in a proctored setting (accomplished through the PostCommit interface), students gain a new motivation for deep C2C understanding (regardless of the source of their submission) and supplemental practice explaining their own ideation and understanding of submitted code (a skill that is often omitted from CS curricula). Likewise, instructors gain a new tool for assessing their class' C2C mastery and providing curated feedback on how students can improve their explanations in interview or conference settings. The PostCommit demo features a sample workflow from student code submission via GitHub Classroom to quiz question curation and automated feedback. The demo also reviews pilot data from a PostCommit deployment in 11 CS courses spanning all grade levels of the curriculum at LMU, including comparisons of student PostCommit quiz scores vs. assignment scores and rates of instructor changes to automated feedback. Data demonstrates large gaps in student C2C mastery when comparing explanatory vs. programmatic correctness.