Challenges to the monitoring of deployed AI systems:

As artificial intelligence (AI) systems are increasingly integrated into commercial and government applications, there is a growing need to monitor these systems in real-world settings. Although pre-deployment evaluations are valuable for assessing AI system capabilities at multiple points prior to release, they are predominantly conducted in controlled testing environments. Post-deployment monitoring is crucial for (1) validating that AI systems operate reliably as expected in real-world scenarios, (2) tracking unforeseen outputs that occur due to, e.g., model non-determinism or dynamic input conditions, and (3) visibility into unexpected consequences of AI systems in deployment contexts. Stakeholders across the AI ecosystem agree on the need for post-deployment monitoring; however, monitoring best practices, validated methodologies, and common terminology are still nascent and scattered across the field. This report proposes monitoring categories and surfaces challenges to robust post-deployment AI system monitoring, rooted in practitioner workshops and a literature review. The identified gaps, barriers, and open questions highlight opportunities for further investigation and innovation. Notably, this report quotes practitioners’ repeated calls for guidance on post-deployment AI system monitoring methods.