YOLOv8-MS-DWConv: enhancing hazardous goods detection in X-ray security imaging with multi-scale depthwise separable convolution

X-ray security inspection plays a vital role in public safety by enabling the automated detection of hazardous and prohibited items. However, real-world inspection scenarios often involve dense object stacking and severe occlusion, leading to frequent missed or false detections and limiting the effectiveness of existing detection models. To address these challenges, we propose a novel object detection framework based on You Only Look Once (YOLO)v8, termed YOLOv8-MS-DWConv. The model integrates a newly designed Multi-Scale Depthwise Separable Convolution (MS-DWConv) module to enhance feature extraction for small and occluded objects while preserving computational efficiency. Furthermore, we introduce a new localization loss function, Pixels-Intersection-over-Union (PIoU), as a replacement for the conventional complete IoU (CIoU) loss, which significantly improves bounding box regression performance, particularly in scenarios involving rotated or overlapping targets. By incorporating both MS-DWConv and PIoU into the YOLOv8 architecture, the proposed model achieves a lightweight yet robust detection capability tailored for complex X-ray security applications. Extensive experiments on three public benchmark datasets—CLCXray, HIXray, and SIXray—demonstrate the superior performance of our approach. Specifically, mAP@0.5:0.95 improved by 2.9% on CLCXray, by 1.2% on HIXray (from 50.0% to 51.2%), and by 4.5% on SIXray (from 61.5% to 66.0%). These results validate the model’s enhanced robustness and generalization capability, particularly under heavy occlusion and cluttered conditions.