Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Research

Read full text
Authors: Nimisha Karnatak, Mohamad Chatila, Daniel Alejandro Pinzon Hernandez, Reza Yazdanfar, Michelle Dugas, Renos Vakis
Cited by: 1
References: 66

Abstract

General-purpose LLMs pose misinformation risks for development and policy experts, lacking epistemic humility for verifiable outputs. We present AVA (AI + Verified Analysis), a GenAI platform built on a curated library of over 4,000 World Bank Reports with multilingual capabilities. AVA’s multi-agent pipeline enables users to query and receive evidence-based syntheses. It operationalizes epistemic humility through two mechanisms: citation verifiability (tracing claims to sources) and reasoned abstention (declining unsupported queries with justification and redirection). We conducted an in-the-wild evaluation with over 2,200 individuals from heterogeneous organisations and roles in 116 countries, via log analysis, surveys, and 20 interviews. Difference-in-Differences estimates associate sustained engagement with 2.4–3.9 hours saved weekly. Qualitatively, participants used AVA as a specialized “evidence engine”; reasoned abstention clarified scope boundaries, and trust was calibrated through institutional provenance and page-anchored citations. We contribute design guidelines for specialized AI and articulate a vision for ‘ecosystem-aware’ Humble AI.

Fields of Study & Subjects

Computational and Text Analysis Methods Ethics and Social Impacts of AI Research Data Management Practices

Publication details

Journal / source
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Pages
1–32
Publication date
13 April 2026