Device Differentiated Protection Using Federated Learning in Heterogeneous IoT Networks

In machine learning (ML), federated learning (FL) has emerged as a solution to privacy concerns by enabling collaborative model training without sharing local data. This study explores the application of FL in the Internet of Things (IoT) and addresses challenges in data distribution and model selection across heterogeneous environments. We propose an FL framework that integrates ensemble learning with stacking and knowledge distillation, and we conduct experiments to evaluate its performance. Our approach involves a stacking ensemble of weak learners on IoT devices, aggregating results with a convolutional neural network (CNN) model on the server, and applying knowledge distillation to improve model performance and defense against malicious attacks. We tested our framework on the N-BaIoT, IoT-23, and ToN-IoT datasets, comparing it with FL algorithms such as FedAvg, FedProx, FedSGD, FedOpt, and FedDyn, as well as the centralized CNN algorithm. The results show competitive accuracy, precision, recall, and F1-score, with notable advantages in model size, inference time, and handling of heterogeneous data among resource-constrained IoT devices. Knowledge distillation further enhances generalization and performance in distributed data scenarios. With heterogeneous IoT devices, our proposed design outperforms the FL algorithms mentioned above. For example, on the ToN-IoT dataset, compared to other FL methods, our method demonstrates that using separate ensemble learning models on IoT devices reduces IoT resource requirements by over 50% while achieving an F1-score that is over 14% higher.