A Vision Linguist Insight System for Multimodal Visual Question-Answering

Video Question Answering (VQA) enables machines to comprehend visual content and respond to natural language queries, bridging the gap between vision and language for real-world tasks. Despite its potential, existing models face challenges in aligning spatio-temporal features with linguistic context, managing cross-modal complexity, and preserving contextual semantics. To overcome these limitations, this research proposes the Vision-Linguistic Insight System (VLIS), a novel framework that integrates attention-driven architectures and multimodal feature representation. It leverages 3D ResNet for capturing temporal video dynamics, DETR for extracting spatial object cues using self-attention, and a BERT encoder for generating context-aware question embedding. Graph Convolutional Networks (GCNs) are used to represent temporal and semantic links by modeling these aspects as a single, cohesive graph. For precise answer prediction, a multi-head attention fusion technique improves cross-modal alignment. VLIS outperforms several cutting-edge techniques with accuracies of 44.8% and 63.4% when tested on the MSRVTT-QA and NExT-QA datasets, respectively. Each component's efficacy is verified by an ablation study. The VLIS framework presents a scalable and interpretable solution, well-suited for applications in robotics, education, and multimedia analytics where complex visual-linguistic reasoning is essential.