RiTeK: A Dataset for Large Language Models Complex Reasoning over Textual Knowledge Graphs in Medicine
Abstract
nowledge Graphs (RiTeK), covering a broad range of topological structures. Specifically, we synthesize realistic user queries integrating diverse topological structures, relational information, and complex textual descriptions. We conduct a rigorous medical expert evaluation process to assess and validate the quality of our synthesized queries. RiTeK also serves as a comprehensive benchmark dataset for evaluating the capabilities of retrieval systems built upon LLMs. By assessing 11 representative retrievers on this benchmark, we observe that existing methods struggle to perform well, revealing notable limitations in current LLM-driven retrieval approaches. These findings highlight the pressing need for more effective retrieval systems tailored for semi-structured data in the medical domain. The dataset is available here.
Fields of Study & Subjects
Semantic Web and Ontologies
Publication details
- Journal / source
- Findings of the Association for Computational Linguistics: ACL 2026
- Publisher
- Association for Computational Linguistics
- Volume
- 2026
- Pages
- 35985–36001
- Publication date
- 1 January 2026
- Language
- EN