OmniBench-RAG: A Multi-Domain Evaluation Platform for Retrieval-Augmented Generation Tools
Abstract
While Retrieval Augmented Generation (RAG) is widely adopted to enhance LLMs, evaluating its true performance benefits in a reproducible and interpretable way remains challenging. Existing methods often fall short: they lack domain coverage, employ coarse metrics that miss sub-document precision, fail to capture computational trade-offs, and suffer from data leakage in some evaluation datasets (leading to unfair assessment results). Most critically, they provide no standardized framework for comparing RAG effectiveness across different models and domains.
Fields of Study & Subjects
Topic Modeling
Recommender Systems and Techniques
Publication details
- Journal / source
- Proceedings of the 2026 IEEE/ACM Third International Conference on AI Foundation Models and Software Engineering
- Publisher
- ACM
- Pages
- 228–232
- Publication date
- 12 April 2026
- Language
- EN