Attention! Rethinking What We Measure in CHIIR Studies

Attention is a crucial construct in Interactive Information Retrieval (IIR) activities and has become an area of growing interest among CHIIR researchers. Despite this importance, explicit definitions are rare. Many studies refer to “attention” without specifying which process or aspect is under examination, and often operationalize it using practical indicators (e.g., eye fixations, dwell time, cursor traces, interaction logs) without a clear conceptual framework guiding measurement choices, raising concerns about the comparability of findings. This paper reviews how attention has been defined, operationalized, and measured in CHIIR publications from the conference’s inception in 2016 through 2025. We searched the ACM Digital Library for "attention" within CHIIR proceedings, finding 296 results. After filtering and using semantic similarity, we narrowed it to 45 relevant papers, from which 19 were selected for in-depth review based on their clear relevance to attention. Drawing on theoretical frameworks and empirical findings from psychology and cognitive science, we analyze how CHIIR researchers define and apply the concept of attention in various contexts. Our review uncovers a variety of interpretations, aspects, and measurement approaches to attention, which reflect broader challenges in bridging cognitive theory and information interaction research. We discuss key issues underlying these differing uses of the term “attention,” and outline possible directions for advancing conceptual clarity and methodological robustness of attention research within CHIIR. We hope this perspective paper raises researchers’ awareness of the need to clearly define attention, thereby promoting greater rigor and reproducibility in future IIR studies.