Evaluating Text-to-Speech for NPC-to-NPC Dialogues in 3D Scenes: A User Study with Video-Rendered Conversations for VR

View at publisher Summarise with VIRAT
Authors: Alexandre C. Ferro Filho, Gustavo H. W. Barbosa, Andressa Araújo Bastos, Letícia Lima Mendes, Augusto Seben da Rosa, Paula Leandra Loeblein Pinto, Gabriel Paes Landim Lucena, Rafael Teixeira Sousa, Arlindo Rodrigues Galvão Filho
Cited by: 0
References: 14

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Recent advances in open-source text-to-speech (TTS) systems promise scalable solutions for populating virtual reality (VR) environments with diverse and interactive non-player characters (NPCs). This paper presents a perceptual user study evaluating seven modern TTS models embedded in video-rendered 3D scenes inspired by VR dialogue contexts. Participants rated nine dimensions of speech quality, including naturalness, emotional adequacy, clarity, fluency, and voice consistency. Results show that intelligibility is solved (clarity >4.5 across all models). Chatterbox achieved the best overall performance, particularly in naturalness and emotional adequacy, with XTTS and FishSpeech also ranking highly. In contrast, SparkTTS, while expressive, suffered from unstable speaker identity. These findings highlight that while clarity is now a baseline, advancing VR requires TTS systems that deliver expressive, consistent, and characterful speech, with models like Chatterbox pointing the way forward. The study does not claim to replicate full in-headset VR experiences, but instead provides an initial proxy for audio-visual dialogue evaluation relevant to immersive applications.

Fields of Study & Subjects

Speech Recognition and Synthesis Speech and dialogue systems Speech and Audio Processing

Publication details

Journal / source
IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)
Pages
243–247
Publication date
26 January 2026
Language
EN