Are CS1 Students More Creative than LLM in Solving a Problem? Preliminary Results on a Comparison of Code Diversity

Large Language Models (LLMs) are increasingly integrated into CS education, with students using them to generate problem solutions. Although LLMs can produce correct solutions, do their approaches align with the diversity of approaches in student code? This work aims to propose an analytical workflow for investigating this topic. We proposed a workflow and evaluated it using 999 student Java code submissions from two questions in a CS1 dataset, along with matched zero-shot solutions generated by OpenAI GPT-4. Zero-shot prompting was chosen to mirror students' typical use of LLMs. Our preliminary results indicate that students consistently demonstrated greater diversity in problem-solving methods than the LLM, especially on more complex tasks. These findings reinforce concerns that heavy reliance on LLMs might reduce students' creativity when solving programming tasks. They also inspire design implications across several future directions, including cheating detection, the use of simulated students, and pedagogy refinement to promote more diverse approaches to students.