Facilitating Non-Verbal Communication for People with Intellectual Disability through Intent Interpretation in Image-Based Interaction

This research explores the use of generic images as a medium for non-verbal communication among individuals with intellectual disabilities. Unlike traditional visual symbols, generic images depicting natural scenes and meaningful life events offer richer contextual cues, making them more intuitive and easier to understand. While existing assistive technologies—such as Augmentative and Alternative Communication (AAC) systems and Visual Scene Displays (VSDs)—primarily emphasize language acquisition, my approach seeks to support broader goals of self-expression, social connection, and group participation. To enhance communication, I leverage image recommendation systems combined with user intent detection to facilitate more accessible and meaningful image selection for communication and clarification. The research adopts a participatory design methodology, involving iterative co-design of prototypes with participants from disability service organizations. Initial exploratory studies examined image-based communication and intent detection using Large Language Models, followed by the development and evaluation of algorithms aimed at improving image accessibility through simulated-user testing. Current work focuses on prototype implementations that investigate image intent interpretation within assistive communication contexts, aiming to identify system requirements and assess the impact on the daily interactions of individuals with intellectual disabilities.