Audo-Sight: Advancing Ambient Interaction for Blind and Visually Impaired Individuals

A new assistive system named Audo-Sight has been developed to enhance ambient interaction for blind and visually impaired (BVI) individuals. Integrating Multimodal Large Language Models (MLLMs), Audo-Sight offers context-aware interactions in both personalized and public settings. In personalized environments, the system adapts to individual user preferences, while in public spaces like museums and shopping malls, it employs a shared architecture that adjusts to the current user without manual reconfiguration. Features such as an Age-Range Determiner and Safe Query Filter ensure appropriate interactions, and NeMo Guardrails maintain respectful responses. By combining multimodal reasoning with BVI-cognizant response editing, Audo-Sight represents a significant advancement in AI-driven accessibility technology, aiming to increase autonomy and interaction for BVI individuals in social settings. (arxiv.org)