Modeling the Dynamics and Impact of Human-AI Interactions with Attention Transition Network Analysis

Network analysis is a well-established approach for the modeling of interactions and relationships across several domains of educational research. However, existing methods face limitations in capturing the temporal order, depth and influence within temporal data. To address these challenges, we introduce Attention Transition Network Analysis (ATNA), a temporal network framework inspired by the attention mechanism in large language models that captures temporal depth where neighboring nodes receive higher attention which decays by distance. The attention and decay can be tuned according to context. Hence, it provides a flexible framework for modeling the temporal depth of relevance. ATNA includes confirmatory testing: bootstrapping to evaluate individual edge stability, case dropping for centrality robustness testing, and permutation for comparing across different processes. Such an extensive level of confirmatory testing affords researchers rigorous mechanisms to ensure that the results of their analysis are genuinely meaningful and not artefacts within the data. In this paper, we introduce the foundations of the ATNA method, compare it to existing methods, describe its mathematical foundations, and illustrate it with a case study in modeling student-AI interactions in programming education.