Large-Scale Automated Auditing of User Feedback in Games: A Resource-Efficient Ontology-Guided Framework for Production Deployment
Abstract
Game developers face critical operational bottlenecks in processing large-scale user feedback to identify actionable design weaknesses. Manual expert analysis is prohibitively expensive ($25.00 ∼ 100.00 per hour), while existing sentiment analysis tools lack the structural granularity required for diagnostic auditing. This paper presents a resource-efficient ontology-guided expert system that automatically decodes player feedback into a validated 28-dimensional design taxonomy via zero-shot inference-requiring only prompt engineering, without model fine-tuning or annotated training data. We processed N = 14, 574 Steam reviews spanning 887 titles using Qwen-2.5-7B on a single consumer GPU (RTX 4080 Laptop) in approximately 40 hours. Multi-architecture validation across three diverse models yielded a mean PABAK = 0.82, indicating high inter-model consistency. The system achieved 96.8% coverage at an average marginal cost below $0.0002 per review-representing an analysis cost below 1/1,000th of manual expert coding. Comprehensive validation through multi-evidence triangulation establishes robustness: bootstrap PCA stability [1] (24 of 28 components for 90% variance), permutation-tested correlations (p < 0.001), and low multicollinearity (mean VIF = 1.08). Comparative baselines confirm substantial advantages over BERT zero-shot and S-BERT clustering. This framework transforms user review analysis from periodic manual audits into continuous, automated operational intelligence-democratizing advanced design analytics for resource-constrained developers.
Fields of Study & Subjects
Sentiment Analysis and Opinion Mining
Digital Games and Media
Computational and Text Analysis Methods
Publication details
- Publication date
- 25 February 2026
- Language
- EN
- Document type
- Preprint