Machine Learning-Based Fault Prediction in Large- Scale Distributed Systems

Cloud computing has piqued the curiosity of both academics and regular people. Google offers global cloud services. The framework detailed in this paper is an ML-based fault prediction system proposed as an ensemble, applied to the Google 2019 Cluster Sample dataset, and provides a depiction of the actual workload execution and resource utilization patterns in large-scale distributed systems. The given methodology follows a systematic workflow that includes data preprocessing, feature engineering, labeling of categorical variables, and addressing class imbalance. XGBoost, LightGBM, and a Voting Classifier are sophisticated ensemble models that are evaluated using the standard performance matrix. The experimental results prove that the Voting Classifier is superior to the individual models with a high accuracy rate of 97.97, high precision and recall as it identifies all the cases of faults. The proposed ensemble framework is superior to existing ML and DL methods, as demonstrated by a comparative analysis. The findings affirm that feature optimization combined with ensemble learning is a powerful, scalable fault-prediction system for distributed systems.