Position: Static Benchmarks and Accuracy-Centric Metrics Are Inadequate for Responsible Foundation Models: A Call for Holistic Evaluation Integrating Risk Factors, Performance, and Governance Frameworks

Abstract This position paper argues that reliance on static benchmarks and accuracy-centric metrics is insufficient for assessing the responsibility of foundation models. High benchmark performance is often treated as a proxy for trustworthiness, yet accuracy alone provides limited insight into critical risks such as bias, privacy leakage, unsafe behavior, loss of trust, and governance non-compliance. We propose a holistic evaluation perspective that jointly considers task performance, risk factors, and governance alignment. This view is formalized through a composite Responsible AI (RAI) formulation and illustrates how models with similar accuracy can differ substantially in real-world acceptability. By examining limitations of current evaluation practices and engaging alternative viewpoints, this paper encourages evaluation approaches that better reflect deployment realities.