Meets Expectations: System Health Analysis and Prognosis for Embedded and Cyber-physical AI

The health management of AI-based systems differs strongly from that of traditional systems, forcing a rethinking and partial reinvention of established techniques: The AI inside provides the capacity to adapt to an operational context – and that leads to a variability in system behaviors that renders conventional health and performance indicators possibly obsolete. Also, even core AI functionality, like perception, can hardly be assessed without complicated reasoning about whether a lack of performance is circumstantial or an actual system health issue. For these reasons, we introduce a system health monitoring methodology that checks health and key performance indicators against expectations while factoring in mitigating circumstances, like environmental effects. This methodology, which is based on probabilistic reasoning, allows the detection of system health degradation and root-cause analytics and is used by us to ensure the operational fitness of safety-critical systems, i.e., Automated Vehicles. As this domain is subject to temporal changes like seasons that impact a system’s performance more than many developing health issues, we combine health monitors with domain monitors and drift detection. Overall, this provides probabilistic health management that looks at expectations, sets of observations, their distributions and their dynamics to determine whether an embedded AI is still fit for its purpose, whether the cyber-physical system embodying it continues to meet the AI’s operational requirements, and whether observations indicate a health or fitness trend that will result in a lack of safety. A key aspect of this novel approach to reasoning about system health is that it addresses unique properties of AI-based systems: It works with the hit-or-miss behavior of AI that occasionally fails even on seemingly comparable inputs, and it can investigate adaptive processes by looking at the health of information flows that define the decision-making of AI-based systems.