Security-Enhanced Adversarially Constrained Multi-Agent Reinforcement Learning for High-Speed Ramp Merging

Deep reinforcement learning (DRL) policies for autonomous driving are often susceptible to distributional shifts and adversarial interactions in mixed traffic, where safety and efficiency objectives often conflict. To address this, we propose a unified system-level framework integrating kinematically realistic adversarial interactions into an adaptive safety-constrained architecture. Specifically, we introduce Adversarial Driving Vehicles (ADVs)—physically homogeneous yet behaviorally aggressive counterparts to cooperative autonomous vehicles (CAVs)—that replace traditional heuristic perturbations with physically feasible, safety-critical maneuvers to systematically expose corner case risks. To counter these threats, we formulate a constrained Markov decision process (CMDP) featuring a novel adversary aware adaptive safety threshold. By dynamically adjusting risk tolerance based on real-time adversarial intensity, this mechanism elegantly mitigates the over-conservatism of rigid constraints without sacrificing extreme-case safety. To solve this CMDP, we propose the Adaptive Adversarial Constrained Multi-Agent Advantage Actor-Critic (AAC-MAA2C) algorithm. Policies are optimized via a curriculum-based Lagrangian dual approach to ensure stable convergence across progressively challenging environments. Empirical results demonstrate that AAC-MAA2C achieves a superior safety–efficiency trade-off, simultaneously slashing average collision rates by 64.1% and increasing speeds by 19.5% against state-of-the-art baselines.