Multi-criteria decision-making framework for ML model selection in optical network fault detection

Optical networks are among the few technologies capable of meeting the massive data transmission requirements of various future applications, including 6G systems and generative AI solutions. Because access to these applications depends on reliable optical network data transmission, accurate fault detection that can degrade the quality of transmission (QoT) is crucial. Several machine learning (ML) models have been employed for fault detection. Accordingly, selecting a deployment model is challenging due to uncertainty about how humans (often network operators) rank performance metrics. Moreover, even if human subjectivity is not an issue, scalability concerns arise when the number of models and performance metrics is massive, necessitating support for decision-making methods. Therefore, this work presents a framework that integrates two multi-criteria decision-making (MCDM) methods, named the Analytical Hierarchy Process (AHP) and the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), to evaluate and select the most suitable ML model for fault detection in optical networks. In the proposed framework, AHP is first used to derive the relative importance (weights) of the performance metrics based on expert judgments, which are then used by TOPSIS to rank the candidate ML models according to their closeness to the ideal solution. Considering multiple performance metrics, the proposed approach addresses the inherent uncertainties in model selection for fault detection in optical networks, particularly in scenarios requiring scalability, where model selection must balance multiple deployment considerations. Ultimately, results leveraging a dataset collected from an optical testbed demonstrate that ranking ML models based on various performance metrics with different levels of importance can provide more realistic and, thereby, accurate information on the best-suited model.