"A Design-Theoretic Taxonomy for Deconstructing Engagement AI Audit Annotation Results Data Table"
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
"This dataset contains two complementary components for analyzing player engagement in interactive game systems: (1) a large-scale corpus of 14,717 Steam user reviews annotated with a custom 28-dimensional binary activation framework, and (2) a smaller reliability audit subset of 201 reviews (67 annotation tasks) designed to validate cross-model consistency.The smaller audit subset (N=201) consists of 67 targeted annotation tasks evaluated by three architecturally distinct large language models (Qwen-2.5-7B, Llama-3.1-8B, Gemma-2-9B) via a triangular consensus protocol. This component computes reliability metrics including Prevalence-Adjusted Bias-Adjusted Kappa (PABAK) and triple-match rates to validate the robustness of the automated coding scheme, ensuring the main corpus annotations reflect consistent, design-theoretic signals rather than model-specific artifacts."
Fields of Study & Subjects
Sentiment Analysis and Opinion Mining
Computational and Text Analysis Methods
Digital Games and Media
Publication details
- Journal / source
- IEEE DataPort
- Publication date
- 28 March 2026
- Language
- EN
- Document type
- Dataset