Interpretable model-driven end-to-end deep learning of geometric shaping for DML-based IM/DD systems

Direct modulation lasers (DMLs) are a low-cost technology for short-reach intensity modulation and direct detection (IM/DD) systems due to their small footprint and low power consumption. However, their performance is fundamentally limited by nonlinear distortion arising from chirp-dispersion interaction. Neural network (NN)-based autoencoders (AEs) offer a promising solution through end-to-end (E2E) optimization, particularly via geometric shaping (GS) that automatically optimizes transmitted signal constellations. Nevertheless, this approach relies on differentiable and accurate channel models, leading to the adoption of NN-based models that require large training datasets. In this Letter, we propose a low-complexity model-driven framework that establishes a surrogate channel based on composite second-order (CSO) distortion theory, enabling AE-based geometric constellation optimization to suppress chirp-dispersion interaction. This approach combines physics-based interpretability with deep learning's adaptive capabilities. Experimental results demonstrate a 1-dB receiver sensitivity improvement for 64-QAM signals after 10-km standard single-mode fiber (SSMF) transmission, confirming the effectiveness of the proposed scheme in mitigating nonlinear distortions.