Optimizing Federated Learning with Particle Swarm Intelligence for Next-Generation Video Surveillance

Video anomaly detection (VAD) is widely used in computer vision and has critical applications in public safety and surveillance. Recent work has addressed this problem by applying centralized deep learning models and, more recently, federated learning (FL) to preserve data privacy. Unfortunately, centralized approaches raise major privacy and scalability concerns, while standard FL methods such as FedAvg show severe performance degradation under non-IID data distributions, which are typical in real-world surveillance. In contrast, we approach aggregation not as naive averaging, but as an optimization problem. Our key idea is to introduce Particle Swarm Optimization-based Federated Learning (PSO-FL), where swarm intelligence principles adaptively weight client contributions based on empirical performance rather than raw data volume. This reduces convergence instability and improves generalization across heterogeneous surveillance environments. This fixes the limitations of FedAvg in handling non-IID data; however, it does not fully eliminate the privacy-performance gap between federated and centralized training. Consequently, we develop a novel PSO-FL framework built on a CNN-LSTM backbone for spatio-temporal feature learning, designed specifically for distributed VAD. While promising, integrating optimizationbased aggregation into FL is non-trivial due to issues of scalability and fairness across clients. Therefore, we further design validation-driven update weighting and semi-supervised mechanisms to ensure robust optimization. We evaluate PSO-FL on three benchmark datasets-UCF-Crime, UCSD Ped2, and UBnormal-and find that it achieves an F1-score of 0.96 and an AUC of 0.97, consistently outperforming FedAvg and, in some cases, even centralized models. The source code will be released upon acceptance.