Agentic Uncertainty Reveals Agentic Overconfidence

Read full text
Authors: Jean Kaddour, Srijan Patel, Gbètondji Dovonon, Leo Richter, Pasquale Minervini, Matt J. Kusner
Cited by: 0

Abstract

Can AI agents predict whether they will succeed at a task? We study agentic uncertainty by eliciting success probability estimates before, during, and after task execution. All results exhibit agentic overconfidence: some agents that succeed only 22% of the time predict 77% success. Counterintuitively, pre-execution assessment with strictly less information tends to yield better discrimination than standard post-execution review, though differences are not always significant. Adversarial prompting reframing assessment as bug-finding achieves the best calibration.

Fields of Study & Subjects

Adversarial Robustness in Machine Learning Ethics and Social Impacts of AI AI-based Problem Solving and Planning

Publication details

Journal / source
arXiv (Cornell University)
Publication date
6 February 2026
Language
EN
Document type
Preprint