Efficient Explainable Metric for Semantic Communication of Images Using Image Captioning

As existing networks are projected to reach the theoretical limit for Shannon capacity, the concept of semantic communication is proposed in literature. In semantic communication the meaning of the message is extracted and transmitted over the network. This work proposes an evaluation metric for a semantic communication system for image transmission. The considered system extracts the semantic information of an image as a caption, transmits the caption over the network and reconstructs a semantically similar image after feeding the caption to an image generation model. The system uses the One-For-All (OFA) image captioning model and the stability diffusion XL image generation model. The system is tested on images from the MS COCO dataset. Evaluation is performed by measuring the similarity between the sent caption and the caption of the generated image at the receiver. The proposed metric is based on a keyword matching similarity score to quantify the semantic similarity between the corresponding captions. The proposed algorithm extracts the keywords from the sent caption using dependency parsing; the keywords and their synonyms are compared with the generated caption to quantify similarity between the captions. Common text similarity metrics are also used, such as ANLS, ROUGE, BLEU, METEOR and BERT sentence similarity. The results indicate that the similarity between the two captions is high, as the values for all the text similarity measures were above 0.6. The considered semantic communication system preserves the semantic information of the transmitted image while reducing the message size and transmission latency by more than 99%.