One-Step Diffusion Distillation Evolutionary Algorithm with Score Implicit Matching for Neural Architecture Search

This paper addresses two persistent challenges in Evolutionary Neural Architecture Search (ENAS): high computational cost and premature convergence. We propose a novel method—One-Step Diffusion Distillation Evolutionary Algorithm with Score Implicit Matching for NAS (OSD2E-SIM-NAS)—which significantly reduces the complexity of the search process while maintaining performance. The key innovation is the distillation of the iterative diffusion-based evolutionary process into a single inference step via Score Implicit Matching (SIM), reducing computational complexity fromO(T) toO(1). SIM leverages implicit score matching to enable dynamic variance modulation, naturally balancing exploration and exploitation—an essential characteristic inherited from diffusion models. To further mitigate premature convergence and improve search robustness, we introduce a diversity-preserving mechanism that combines simulated annealing, stochastic teacher sampling, and periodic injection of new individuals. We validate the portability and effectiveness of OSD2E-SIM-NAS across both image classification and generative tasks. The method achieves test error rates of 2.55% on CIFAR-10, 16.15% on CIFAR-100, and 24.61% on ImageNet. In generative tasks on CIFAR-100, it attains an Inception Score (IS) of 9.01± 0.07, a Fréchet Inception Distance (FID) of 15.06, and a remarkably low search cost of 0.39 GPU-days, demonstrating superior efficiency, diversity, and adaptability across tasks.