Importance-aware image segmentation-based semantic communication for autonomous driving

This article studies the problem of image segmentation-based semantic communication in autonomous driving. In real traffic scenes, the detecting of objects (e.g., vehicles and pedestrians) is more important to guarantee driving safety, which is always ignored in existing works. Therefore, we propose a vehicular image segmentation-oriented semantic communication system, termed VIS-SemCom, focusing on transmitting and recovering image semantic features of high-important objects to reduce transmission redundancy. First, we develop a semantic codec based on Swin Transformer architecture, which expands the perceptual field thus improving the segmentation accuracy. To highlight the important objects' accuracy, we propose a multi-scale semantic extraction method by assigning the number of Swin Transformer blocks for diverse resolution semantic features. Also, an importance-aware loss incorporating important levels is devised, and an online hard example mining (OHEM) strategy is proposed to handle small sample issues in the dataset. Finally, experimental results demonstrate that the proposed VIS-SemCom can achieve a significant mean intersection over union (mIoU) performance in the SNR regions, a reduction of transmitted data volume by about 60% at 60% mIoU, and improve the segmentation accuracy of important objects, compared to baseline image communication.