Dialogs with GenAI NPCs: Exploring Player Interactions with Speech Agents in a VR Game

Generative artificial intelligence (GenAI) can potentially enhance player experiences by enabling more dynamic and adaptive interactions. This exploratory research investigates player perceptions and interactions with GenAI-based non-player characters (NPCs) in a speech-based Virtual Reality (VR) game. “Office Whispers,” is a VR adventure-puzzle game featuring four GenAI-based NPCs with diverse human characteristics. Our findings from the user-study show that players had an overall positive experience and found the game novel. Players appreciated the freedom of expression and felt deeply immersed when NPCs responded in a believable manner. However, issues such as unnatural conversational flow, incorrect or inconsistent responses, and uninformative dialogue disrupted immersion and gameplay. Our study contributes new insights into the potential and the current limitations of GenAI-based NPCs in speech-based games. We outline design implications and offer guidance for game designers and developers on adapting AI-based NPCs to support more immersive and engaging player experiences. Human-centered computing → Natural language interfaces;Applied computing → Computer games. Human-centered computing → Natural language interfaces; Applied computing → Computer games.