A Benchmark for Citation-Grounded Geographic QA

GroundedGeo Updated release of GroundedGeo, a benchmark for citation-grounded, freshness-aware, and conflict-aware geographic QA. Dataset 200 queries across 5 hard-case buckets (40 each): Boundary-adjacent queries (near county/state lines) Ambiguous name queries (place names in 10+ states) Overlapping jurisdiction queries Freshness-sensitive queries Conflict queries (official vs. community sources) 147 dev / 53 frozen test split 100% evidence coverage: every query includes gold_evidence with URLs, source types, retrieval dates, and publication dates 85 multi-source queries for testing source preference & conflict handling Frozen Benchmark Results (Test Split) System Accuracy Conflict Handled Closed-Book LLM 17.0% 11.1% Naïve RAG 79.2% 11.1% Official-First RAG 94.3% 100.0% Freshness-Filter RAG 79.2% 11.1% Conflict-Aware RAG 92.5% 88.9% Key Finding: Naïve RAG reaches 79.2% accuracy but fails on conflicting sources (11.1% conflict-handled). Adding official-source ranking improves overall accuracy to 94.3% and raises conflict handling to 100%. Artifacts: groundedgeo_paper_v2.pdf — Full research paper data/groundedgeo_v1.0.json — Full dataset (schema + metadata) data/groundedgeo_v1.0.jsonl — HuggingFace format data/groundedgeo_v1.0.csv — Spreadsheet view results/final_results.json — Frozen test metrics results/error_analysis.json — Error analysis Reproducibility Run baselines + evaluation from the repo (eval/) using the included splits and configs. How to Cite APA: Pandya, N. (2026). GroundedGeo: A Benchmark for Citation-Grounded, Freshness-Aware, and Conflict-Aware Geographic QA. Zenodo. https://doi.org/10.5281/zenodo.18142378 BibTeX: @misc{pandya2025groundedgeo, title={GroundedGeo: A Benchmark for Citation-Grounded, Freshness-Aware, and Conflict-Aware Geographic QA}, author={Pandya, Nidhi}, year={2026}, doi={10.5281/zenodo.18142378}, url={https://github.com/nidhip1611/GroundedGeo}, publisher={Zenodo} } Links HuggingFace: https://huggingface.co/datasets/nidhip1611/groundedgeo GitHub : https://github.com/nidhip1611/GroundedGeo Author Nidhi Pandya Pace University, Seidenberg School of Computer Science ORCID: 0009-0001-7313-5892 License Dataset: CC BY 4.0 Code: MIT License