Deep learning-based railway intrusion detection model under complex environments

Abstract Detecting railway foreign object intrusion under complex conditions presents significant challenges due to limited illumination subtle defect features. In this study, we propose solution of YOLOv12. To support training and evaluation, we construct a foreign object intrusion data set. Building upon the YOLOv12 architecture, we introduce a Boundary Aggregation Unit (BAU) and design a feature extraction module based on a dynamic hyperbolic tangent activation function and a Feature Feedforward Network (FFN). By effectively reduces the impact of complex backgrounds, thereby resolving the feature extraction challenges for railway intrusion detection in such environments. Second, to tackle the difficulty of anchor box localization with traditional loss functions in complex environments, we replace CIoU with Powerful-IoUv2 (PIoU2). By introducing a size-adaptive penalty factor and a non-monotonic attention mechanism, the detection accuracy of the model in complex environments is further enhanced. The result demonstrate that proposed model improved mAP from 0.854 to 0.895