Deference-Aware Evaluation for Human-in-the-Loop AI Systems
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Standard AI evaluation frameworks penalise calibrated uncertainty — creating systematic pressure toward overconfident AI in exactly the contexts where human oversight matters most. This paper proposes Deference-Aware (DA) Evaluation: the first unified framework for HITL AI classifiers that credits appropriate deferrals as correct rather than penalising them. Instantiated in systematic review evidence synthesis across four evaluation tiers and nine frontier models. Yuyu Shen | Hopperlace Research | March 2026 Related product: https://www.evidencesynthesisai.com
Fields of Study & Subjects
Ethics and Social Impacts of AI
Human-Automation Interaction and Safety
Explainable Artificial Intelligence (XAI)
Publication details
- Publication date
- 27 March 2026
- Language
- EN
- Document type
- Preprint