Lightweight Techniques for Federated Anomaly Detection in Log Data

Accurately and efficiently identifying anomalies within log data is crucial for maintaining the reliability, availability, and security of modern computing systems. In interconnected environments, log data often come from distributed sources such as Internet of Things (IoT) devices, industrial networks, or smart grids. Centralizing these logs for anomaly detection can be challenging due to strict confidentiality requirements, regulatory constraints, and the limited bandwidth of edge networks. Federated learning (FL) offers an alternative by enabling local model training on site, while aggregating only models instead of sensitive data, thereby preserving data confidentiality and reducing data transfer. This paper develops and evaluates a federated log-anomaly detection pipeline and analyzes its components. We adapt lightweight anomaly detection techniques in a federated setting, comparing them with deep learning (DL) methods, assessing detection capabilities, computational efficiency, memory requirements, and inference time. We explore the residual privacy risks in FL within the proposed pipeline, develop a threat model and suggest mitigation strategies. Our findings indicate that lightweight anomaly detection methods can match the effectiveness of DL techniques in the FL framework, while often providing improved computational and communication efficiency. Furthermore, these techniques show better resilience to non-IID data distributions, a typical FL challenge that can severely hinder the effectiveness of traditional machine learning models. However, the memory footprint of lightweight models varies with dataset characteristics and, in some cases, may exceed that of DL models.