AI-based wearable vision assistance system for the visually impaired: Integrating real-time object recognition and contextual understanding using large vision-language models

Visual impairment affects the ability of people to live a normal life and perform daily life activities. Traditional approaches to helping visually impaired people are deficient in obtaining contextually rich environmental information. This research introduces a novel large vision language model (LVLM) based wearable vision assistance system with a hat-mounted camera connected to a Raspberry Pi 4 Model B to recognize new people or objects through a one-click process. The system allows users to add data on new individuals and objects for later recognition, enhancing the accuracy of the recognition over time. LVLM enables richer environmental descriptions compared to traditional CNN-based systems. The proposed system achieved 90% object recognition accuracy in well-lit conditions and 85% average usability score from users participated in usability study, indicating high reliability of the system and users satisfaction. Additionally, the proposed system incorporates a distance sensor that activates a beeping sound using a buzzer as soon as the user is about to collide with an object, helping to ensure safety while navigating their environment. A comprehensive comparative study is carried out to evaluate the proposed system against traditional support techniques. Comparative analysis shows that the proposed system demonstrates a promising approach for integrating contextual understanding and personalization in wearable assistive technologies and solves the major issues faced by the community of visually impaired people. This solution has practical use in environments such as smart shopping, urban navigation, and public transportation systems, making it adaptable to both personal and smart city contexts.