You Only Transmit Once: Unified Generation and Comprehension for Efficient Semantic Communication

Semantic communication (SC) systems face the challenge of completing generation and comprehension tasks simultaneously under high data compression. To address this issue, this paper proposes a You Only Transmit Once (YOTO) SC system, which achieves efficient performance by synergistically integrating images and texts. For generation tasks, YOTO integrates a multiple granularity visual encoder, a digital modulation autoencoder, and a conditional probability-based visual decoder. These parts collectively ensure high-fidelity image reconstruction and accurate object detection. Meanwhile, YOTO utilizes the Source Feature Selection (SFS) and Channel Feature Selection (CFS) module for joint compression of source and channel coding data. To achieve a globally optimal compression, the Libra module dynamically adjusts feature retention ratios between CFS and SFS. Regarding comprehension tasks, YOTO eliminates the need for additional text data transmission by simply replacing the visual decoder with a Multimodal Language Model that incorporates the visual causality and bridging module. This architecture effectively converts demodulated visual data into descriptive captions while enabling emotion-aware text generation through prompt engineering. Extensive simulation experiments demonstrate the superior performance of YOTO in diverse channel conditions. In generation tasks, the YOTO system achieves PSNR values exceeding 30 dB both in the AWGN and Rayleigh fading channels when the SNR is 6 dB. While in comprehension tasks, it attains sentence similarity scores above 0.83 at 10 dB SNR. YOTO system effectively resolves cross-modal semantic disparities, thereby paving the way for future developments of SC systems that can balance generation and comprehension tasks.