SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Search

Recent advances in Neural Networks (NNs) call for innovative architecture designs that can balance performance and efficiency. Hardware-aware Neural Architecture Search (HW-aware NAS) has emerged as a promising approach for automating NN design through multi-objective optimization, such as evolutionary algorithms. However, the complex relationship between NN design parameters and HW-aware NAS optimization objectives remains largely underexplored, limiting the ability to exploit this knowledge to effectively guide the search. Moreover, the significant amount of evaluation data generated during the search is underutilized, despite its potential to refine optimization strategies and improve Pareto front approximation. We propose SONATA, a self-adaptive evolutionary algorithm for HW-aware NAS. SONATA dynamically adjusts its evolutionary operators by learning the relative importance of NN design parameters. We employ tree-based surrogate models and a reinforcement learning agent to learn how and when to evolve neural architectures. Evaluations across multiple NAS search spaces and hardware platforms on the ImageNet-1k dataset demonstrate the effectiveness of SONATA, achieving up to ~0.25% improvement in accuracy and up to ~2.42x reductions in latency and energy consumption while attaining up to approximately ~93.6% Pareto dominance over the baseline NSGA-II.