Evaluating Agentic Artificial Intelligence: A Comprehensive Survey of Metrics, Benchmarks, and Methodologies

Agentic Artificial Intelligence (Agentic AI) refers to autonomous systems capable of multi-step planning, subgoal generation, tool use, adaptation, and sustained decisionmaking. As such systems are increasingly deployed in robotics, scientific discovery, software engineering, and large language model (LLM)-based agents, the need for rigorous and multidimensional evaluation has become critical. However, existing evaluation practices, largely inherited from supervised learning and short-horizon reinforcement learning, fail to capture essential agentic properties such as long-horizon coherence, robustness under uncertainty, autonomy, safety, and adaptive behavior. This survey presents a structured and comprehensive analysis of evaluation methodologies for Agentic AI. We introduce an elevendimensional taxonomy encompassing task success, output quality, planning and reasoning, decision-making under uncertainty, tooluse effectiveness, autonomy and human intervention, learning and adaptation, safety and alignment, efficiency and resource utilization, multi-agent collaboration, and long-term memory consistency. Using this taxonomy, we systematically examine the growing ecosystem of benchmarks, frameworks, and evaluation tools, highlighting how they assess agent interactions, behavioral trajectories, and long-horizon performance across reinforcement learning agents, LLM-based agents, and multi-agent systems. Building on this analysis, we identify persistent evaluation gaps, including non-determinism, limited reproducibility, benchmark overfitting, incomplete safety assessment, and insufficient longhorizon evaluation. We conclude by outlining future research directions toward standardized, interpretable, and reliabilityfocused evaluation frameworks that more accurately reflect the real-world capabilities, limitations, and risks of Agentic AI.