AES-CTR–based modulation-layer encryption compatible with high-speed coherent optical transport

We propose a modulation-layer encryption scheme for high-throughput coherent optical systems by extending the Advanced Encryption Standard (AES) with counter (CTR) mode to a symbol-block cipher operating in the modulation domain. AES keystream bits are mapped to per-symbol phase shifts, preserving constellation amplitude and ensuring full compatibility with standard coherent DSP. Independent- and shared-key strategies for multi-subcarrier transmission are analyzed. Simulations for 100-Gsymbol/s DP-4QAM show no OSNR penalty at the forward error correction threshold, while experiments with 96-Gsymbol/s two-subcarrier transmission over 10 ROADM nodes confirm no measurable Q² penalty. The results demonstrate that AES-CTR symbol-block encryption provides scalable modulation-layer encryption with negligible performance impact.