Designing and Prompt-Engineering Spatial Assistants for Immersive Human-AI Interaction
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
This work presents design principles and promptengineering strategies for developing spatially aware Artificial Intelligence (AI) assistants in immersive environments. Focusing on the integration of AI and Virtual Reality (VR), we explore how multimodal, spatially grounded AI assistants can support humans in VR contexts. Drawing from literature in immersive Human-AI Interaction (HAX), we identify key requirements for egocentric, spatial, and contextual awareness. These insights informed the development of Novy, a Large Language Model (LLM)-based AI assistant prototype capable of spatial interactions in virtual environments through voice and gesture commands. Early evaluations indicate that Novy enhances high-level spatial interactions and user satisfaction, revealing design challenges and informing future work toward comprehensive design guidelines for spatially grounded and immersive AI assistants.
Fields of Study & Subjects
Interactive and Immersive Displays
Virtual Reality Applications and Impacts
Social Robot Interaction and HRI
Publication details
- Journal / source
- IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)
- Pages
- 213–218
- Publication date
- 26 January 2026
- Language
- EN