Enhancing Accessibility for Visually Impaired Individuals: Audio-assisted Curry Recognition System

This paper presents a mobile application that assists visually impaired individuals in identifying different curry dishes on a plate through real-time computer vision and audio feedback. The system combines a smartphone camera interface with a deep learning (DL) model and YOLOv8 for object detection and provides results via voice output for an eyes-free user experience. A user-centered design methodology guided development, emphasizing intuitive camera and voice interactions. A lightweight YOLOv8 model (custom-trained to recognize seven types of curries) from a ~700–1.000-image dataset (≈ 100–150 images/class) and a client–server architecture is used to capture meal video and quickly analyze it to return curry names spoken using text-to-speech. During user evaluations with a 20-participant usability study (ages 16–25, 27–40, over 40; spanning profound/near-total low vision and total blindness; routine smartphone users), the system achieved ~ 92.5% accuracy in curry recognition and showed robust performance with response times of a few hundred milliseconds. Usability testing showed an average System Usability Scale (SUS) score of 82/100 (rated “excellent”) and a Net Promoter Score (NPS) of +70, indicating strong user recommendation of the app. All participants independently identified curry and rated audio feedback clarity as 4.4/5. These results show audio-assisted curry recognition. This solution increases dining autonomy for visually impaired users and shows how advanced object detection and interface design can bridge assistive technology gaps. Expanding food categories, improving real-time performance, and considering visual impairment deployment are the future directions.