Effects of Immersion Fidelity and Multimodal Support on Cognitive Load, User Experience, and Behavioral Intentions in Embodied Conversational Agent Interactions

ABSTRACT This study examines how immersion fidelity (2D vs. VR) and multimodal support shape user experience during 20+ min diabetes‐related conversations with an embodied conversational agent (ECA). Using a between‐subjects design, participants engaged in conversation‐only interactions or conversations augmented with subtitles, visualizations, or both. We assessed memory retention, diet attitude change, behavioral commitment intentions to adopt healthier eating habits, perceived workload, working alliance, and subjective impressions of the agent. In conversation‐only interactions, 2D supported stronger working alliance, while VR elicited stronger behavioral commitment intentions. When subtitles and/or visualizations were present, working alliance and behavioral commitment intentions converged across modalities, although subjective impressions remained condition‐specific. Across conditions, VR was associated with higher perceived workload, including consistently higher physical demand and additional increases in mental and overall workload under multimodal support. Memory retention and diet attitude change did not differ between 2D and VR. These findings highlight tradeoffs between relational alignment, motivational impact, and interaction effort when designing scalable ECA‐mediated health conversations across 2D and VR platforms.