Enhancing autonomous vehicle resilience: Roadside sensor networks for robust perception and decision-making in challenging environments

Autonomous vehicles face critical perception limitations in adverse weather and dense traffic, where onboard sensors frequently deliver unreliable data. This paper proposes a novel roadside sensor infrastructure communication framework enhanced by AI foundation models to comprehensively address these challenges. The system integrates infrastructure-based sensors including cameras, LiDAR, radar, and weather stations with context-adaptive fusion techniques that dynamically assess and weight sensor reliability in real time. Key innovations include fail-operational perception capability even under onboard sensor degradation, V2X-optimized data fusion that expands the field of view, and predictive decision-making supported by foundation models. Validation in the CARLA simulation environment demonstrates significant performance improvements: about 35% reduction in occlusion errors, an increase in object detection accuracy to 74.7% by camera-LiDAR fusion approach (compared to 58.3% for cameras alone), and a 28% reduction in collision rates during heavy rain supported by roadside sensor units. By bridging the limitations of onboard perception with resilient infrastructure support and intelligent fusion, this work establishes a foundation for robust, context-aware autonomous driving systems capable of safe operation in complex real-world environments.