Scalable Multilingual Retrieval of Lecture Moments Through Whisper-Based Transcripts and Vector Search
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Educational video platforms host vast collections of long, speech-centric lectures, yet most search interfaces remain metadata-driven and return results at the video level, making it difficult for learners—especially non-English speakers—to locate the exact moment where a concept is explained. This paper presents VideoRAG, a multilingual, timestamp-level video segment retrieval framework for English educational videos using queries expressed in diverse languages. VideoRAG first applies OpenAI Whisper to generate time-aligned transcripts with fine-grained timestamps. The transcript is then chunked and embedded at the sentence/segment level, and the resulting vectors are indexed using a FAISS-based approximate nearest-neighbor database for scalable similarity search. At query time, multilingual user queries are encoded into language-agnostic semantic embeddings using Google Gemini 1.5 and matched directly against the indexed transcript vectors, enabling cross-lingual retrieval without relying on metadata or explicit query translation. The system returns the top-ranked segments together with their corresponding timestamps, allowing users to jump directly to the relevant lecture moments; an optional generation step can further summarize retrieved evidence with timestamp citations. Experiments on real-world lecture datasets show that VideoRAG achieves consistent multilingual retrieval performance with interactive latency, substantially improving segment-level accuracy and precision compared with keyword-based and vision-only baselines. These results demonstrate VideoRAG as a practical foundation for scalable, accessible, multilingual navigation of long-form educational videos.
Fields of Study & Subjects
Video Analysis and Summarization
Information Retrieval and Search Behavior
Topic Modeling
Publication details
- Journal / source
- IEEE Access
- Volume
- 14
- Pages
- 49225–49235
- Publication date
- 1 January 2026
- Language
- EN