Agentic Uncertainty Reveals Agentic Overconfidence
Abstract
Can AI agents predict whether they will succeed at a task? We study agentic uncertainty by eliciting success probability estimates before, during, and after task execution. All results exhibit agentic overconfidence: some agents that succeed only 22% of the time predict 77% success. Counterintuitively, pre-execution assessment with strictly less information tends to yield better discrimination than standard post-execution review, though differences are not always significant. Adversarial prompting reframing assessment as bug-finding achieves the best calibration.
Fields of Study & Subjects
Adversarial Robustness in Machine Learning
Ethics and Social Impacts of AI
AI-based Problem Solving and Planning
Publication details
- Journal / source
- arXiv (Cornell University)
- Publication date
- 6 February 2026
- Language
- EN
- Document type
- Preprint