AI Vision Assistance for Visually Impaired Individuals

View at publisher Summarise with VIRAT
Authors: Samuel Raj E, Thamizharasan M, Saraswathi. T
Cited by: 0
References: 12

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

While there have been many advances made in assistive technology, the gap in providing a cohesive, costefficient, mobile-native, multimodal method to allow independent mobility and situational awareness for people who are visually impaired still exists. Most current systems have limitations, as many systems only provide unimodal capability (such as just being able to detect obstacles), and/or they require the use of specialty, expensive hardware. This paper introduces an approach to creating a Unified Multimodal AI Visual Assistant that can run on any standard smartphone, combining real-time object detection, navigation help, and visual context-based question answering. The created architecture employs OpenCV to assist with fast frame handling, YOLOv8 models to optimize speed for object detection, and a ViLT Transformer for more precise scene interpretation. For an effective fully offline (no internet connection required) speech-to-text recognition and text-tospeech, we combined VOSK and pyttsx3. The performance of the system was tested using both standard data sets (COCO and Open Images) and controlled navigation situations. Metrics tested include mean average precision (mAP), precision, recall,$\mathbf{F 1}$-score, latency, and user feedback. Results are consistent with the testing conducted and exhibit$>95 \%$accuracy with real-time response latency of$\mathbf{1 5 0}-\mathbf{2 3 0 m s}$per frame on generic hardware. By providing a smartphone-only design as opposed to twentieth-century technology which required additional physical devices like wearable sensors or special gloves, this solution creates an easier access point for users. In addition to ease of use, the ability to combine vision with speech and language creates an innovative and versatile product offering for visually impaired individuals, thus providing increased capability and situational awareness.

Fields of Study & Subjects

Assistive Technology in Communication and Mobility Advanced Neural Network Applications Tactile and Sensory Interactions

Publication details

Journal / source
International Conference on Communication, Computing and Electronics Systems (ICCCES)
Pages
1665–1671
Publication date
21 January 2026
Language
EN