E2ETrADS: end-to-end transformer based autonomous driving system for adverse weather conditions

Abstract Adverse weather conditions, such as snow, heavy rain, fog, or limited illumination, pose significant challenges for autonomous vehicles (AVs) by degrading sensor reliability. This paper introduces E2ETrADS, an end-to-end transformer-based autonomous driving framework designed to operate robustly under such conditions. A comprehensive dataset was generated using the CARLA simulator, encompassing both nominal and adverse weather scenarios. The model is trained via imitation learning from an expert driver equipped with weather-adaptive MPC planner and PID controllers, enabling robust control under perception uncertainty. Experimental results demonstrate that E2ETrADS outperforms the TransFuser baseline in adverse conditions, exhibiting fewer infractions and improved lane adherence. The system dynamically adjusts vehicle speed to maintain control stability and adapts its control policies in response to sensor degradation, resulting in fewer missed turns and reduced lane invasions. Furthermore, E2ETrADS generalizes effectively to safety-critical long-tail scenarios, demonstrating human-like reasoning capabilities.