Event-Grounded Question Answering over Long Audio via Structured Retrieval

Read full text
Authors: Kartik Hegde, Arvind Krishna Sridhar, Naveen Vakada, Yinyi Guo, Erik Visser
Cited by: 0

Abstract

Answering natural-language questions over multi-hour audio requires reliable event recognition, temporal grounding, and efficient retrieval. We present LA-RAG (Long Audio Retrieval-Augmented Generation), a structured framework that converts audio into timestamped event records, stores them in an event database, and answers questions using intent-aware retrieval and LLM-based generation. LA-RAG supports two deployment settings: offline grounding mode, in which long recordings are pre-indexed for low-latency querying, and inference-time grounding mode, which performs query-conditioned grounding over shorter, open-ended clips. We evaluate LA-RAG on controlled 24-hour Home-IoT and Industrial-IoT benchmarks and on CASTELLA-QA, derived from real-world audio recordings. The results demonstrate effective long-audio question answering with low query-time latency after grounding and indexing. They also reveal a substantial gap between event detection and temporal localization in current large audio-language models, while showing that explicit timestamped grounding consistently improves temporal reasoning. These findings establish structured grounding and retrieval as a practical complement to generative audio-language models for deployment-oriented long-audio understanding.

Fields of Study & Subjects

Music and Audio Processing Topic Modeling Speech Recognition and Synthesis

Publication details

Journal / source
arXiv (Cornell University)
Publication date
16 February 2026
Language
EN
Document type
Preprint