Exploring the Use of Clustering Algorithms and LLMs to Identify Programming Strategies

In programming courses, students may use various strategies to solve the same problem. Understanding these strategies can be important for teachers and instructors to assess student progress and provide targeted feedback. Traditional clustering methods have been widely used to group similar programming solutions. However, these techniques often rely on syntactical similarities that do not always reflect the strategies for solving a problem. This research uses clustering algorithms and Large Language Models (LLMs) to identify programming strategies. We conducted the experiments using a dataset of correct student solutions collected from ten Algorithms and Data Structures classes at Federal University of Amazonas. Although the Mean Shift and Affinity Propagation clustering algorithms provided us with visually well-separated clusters, quantitative results showed that the algorithms were not accurate in grouping strategies. In contrast, LLMs demonstrated a better ability to identify strategies aligned with human labels. The results suggest that LLMs can be valuable tools to assist programming instructors in analyzing student solutions.