Designing and Prompt-Engineering Spatial Assistants for Immersive Human-AI Interaction

View at publisher Summarise with VIRAT
Authors: Suhana Biswas, Yücel Uzun, Leif Oppermann, Ifigeneia I. Mavridou
Cited by: 0
References: 17

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This work presents design principles and promptengineering strategies for developing spatially aware Artificial Intelligence (AI) assistants in immersive environments. Focusing on the integration of AI and Virtual Reality (VR), we explore how multimodal, spatially grounded AI assistants can support humans in VR contexts. Drawing from literature in immersive Human-AI Interaction (HAX), we identify key requirements for egocentric, spatial, and contextual awareness. These insights informed the development of Novy, a Large Language Model (LLM)-based AI assistant prototype capable of spatial interactions in virtual environments through voice and gesture commands. Early evaluations indicate that Novy enhances high-level spatial interactions and user satisfaction, revealing design challenges and informing future work toward comprehensive design guidelines for spatially grounded and immersive AI assistants.

Fields of Study & Subjects

Interactive and Immersive Displays Virtual Reality Applications and Impacts Social Robot Interaction and HRI

Publication details

Journal / source
IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)
Pages
213–218
Publication date
26 January 2026
Language
EN