Scalable Multilingual Retrieval of Lecture Moments Through Whisper-Based Transcripts and Vector Search

Read full text Summarise with VIRAT
Authors: Lazim Afraz, M Aravind, Faheem Farooq, Muchenedi Hari Kishor
Cited by: 0
References: 11

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Educational video platforms host vast collections of long, speech-centric lectures, yet most search interfaces remain metadata-driven and return results at the video level, making it difficult for learners—especially non-English speakers—to locate the exact moment where a concept is explained. This paper presents VideoRAG, a multilingual, timestamp-level video segment retrieval framework for English educational videos using queries expressed in diverse languages. VideoRAG first applies OpenAI Whisper to generate time-aligned transcripts with fine-grained timestamps. The transcript is then chunked and embedded at the sentence/segment level, and the resulting vectors are indexed using a FAISS-based approximate nearest-neighbor database for scalable similarity search. At query time, multilingual user queries are encoded into language-agnostic semantic embeddings using Google Gemini 1.5 and matched directly against the indexed transcript vectors, enabling cross-lingual retrieval without relying on metadata or explicit query translation. The system returns the top-ranked segments together with their corresponding timestamps, allowing users to jump directly to the relevant lecture moments; an optional generation step can further summarize retrieved evidence with timestamp citations. Experiments on real-world lecture datasets show that VideoRAG achieves consistent multilingual retrieval performance with interactive latency, substantially improving segment-level accuracy and precision compared with keyword-based and vision-only baselines. These results demonstrate VideoRAG as a practical foundation for scalable, accessible, multilingual navigation of long-form educational videos.

Fields of Study & Subjects

Video Analysis and Summarization Information Retrieval and Search Behavior Topic Modeling

Publication details

Journal / source
IEEE Access
Volume
14
Pages
49225–49235
Publication date
1 January 2026
Language
EN