A Comparative Mathematical Analysis of Elliptic-Curve–Enhanced AES for Image Encryption Using Dynamic Keys and S-Boxes

In this study, an enhanced version of the Advanced Encryption Standard (AES) is proposed by integrating Elliptic Curve Cryptography (ECC) to generate dynamic S-boxes and encryption keys. The proposed scheme aims to enhance security while preserving computational efficiency in image encryption. Three NIST- recommended elliptic curves—P-256, P-384, and P-521 are employed to investigate the impact of curve size and algebraic properties on encryption strength and randomness. Experimental results demonstrate that the ECC-enhanced AES scheme achieves strong overall performance, with all configurations enabling perfect image reconstruction without distortion. Entropy analysis confirms a high degree of randomness, particularly when using CTR mode and the AES–ECC (P-521) configuration. Security evaluation shows strong resistance to differential and statistical attacks, as evidenced by high NPCR and UACI values and near-zero correlation coefficients. The combination of CTR mode and ECC provides the most robust protection. Histogram and visual analyses further verify effective concealment of perceptual information, with CTR mode nearly eliminating spatial pixel correlations. Compared with standard AES, the proposed AES–ECC approach, especially with the AES–ECC (P-521) configuration, exhibits improved pixel distribution uniformity and reduced residual visual patterns. Memory usage remains constant across all configurations. Although integrating CTR mode and ECC increases computational overhead, memory usage remains constant across all configurations, and ECB mode with the AES–ECC (P-256) configuration for grayscale images achieves competitive encryption times, reflecting the efficiency of this setup compared to other variants, and visual obfuscation while maintaining acceptable computational efficiency and image quality.