Beyond the Prompt: How Novices Engage with AI During Code Summarization

Generative AI tools are transforming software development. While these tools offer benefits for developers, there are growing concerns regarding their use. One of the main concerns is that students' over-reliance on generative AI can lead to the erosion of foundational software development skills. In this paper, we present the first eye-tracking study investigating how generative AI affects novice programmers' code comprehension during code summarization tasks. Our findings reveal that AI assistance shifts where and how often programmers focus on specific code elements, as well as the depth of their engagement across semantic categories such as arguments, loops, and method calls. AI assistance also led to consistently longer summaries across experience levels, even though it did not significantly affect perceived or automated measures of summary quality. Additionally, we also observed that the less experienced novices tended to use direct summary requests as opposed to code clarification and summary evaluations, highlighting the need to teach AI literacy and responsible tool use in software engineering education.