Co-Evolutionary Mechanism Design for Federated Traffic Classification in Multi-ISP Edge–Cloud Clusters

Abstract Federated learning enables multiple Internet service providers to collaboratively train traffic classification models without sharing raw traffic traces, which is attractive for privacy and business confidentiality in edge--cloud cluster deployments. However, incentive mismatches can motivate strategic behaviors such as low-effort training, selective participation, or subtle update bias, which degrade classification accuracy, fairness, and system stability. This paper proposes a co-evolutionary mechanism design framework that jointly optimizes coordinator policies and adaptive strategic Internet service provider behaviors. A population of coordinator mechanisms, including participation control, reward and penalty rules, auditing policies, and aggregation settings, is evolved via multi-objective search to balance classification utility with communication and computational overhead. In parallel, a population of strategic behaviors is evolved and expanded using a large language model that generates realistic tactics from observed system feedback. Experiments on federated traffic classification show that, under \((p_{\mathrm{str}}=0.4)\), the proposed approach achieves a macro-F1 of \((0.876)\) and improves the worst-ISP accuracy to 82.1%, outperforming fixed-rule and robustness-only baselines while maintaining practical coordination overhead.