Explainability and Reliability Analysis of Generative AI Models for Medical Device Failure Prediction

Medical devices are vital in patient safety and when they fail, the medical treatment might be undermined and legal suits might be filed. In the recent past, Generative AI-based models like Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Diffusion Models, have demonstrated potential in forecasting early failures by identifying patterns of operations and irregularities that are difficult to identify by humans. The black-box properties of these models, however, make them less interpretable and less reliable to take up in safety critical application.The current work hypothesizes a systematic framework in explainability and reliability analysis of generative AI models used in prediction of medical devices failure. The framework combines the most recent feature attribution methods, including saliency maps and counterfactual explanations, to both give interpretable explanations about model predictions as well as assess reliability by measuring robustness, uncertainty quantification and fault-tolerance. Simulated and real-world medical device operational dataset experiments have shown that explainable generative models are able to provide high predictive performance, and also provide clinicians and device operators with actionable insights.Significant contributions are (i) comparative analysis of GAN, VAE, and Diffusion Models based on their predictive performance and interpretability, (ii) reliability assessment methodology in medical device data and (iii) practical suggestions of integrating explainable AI in clinical predictive maintenance processes. These results demonstrate that there is a possibility of reliable AI-based surveillance in medical device environments, which will open the field of safer, proactive device management.