Meta-learning for automated hyperparameter optimization in neural networks: a comprehensive survey

With the breakthrough progress of deep learning across multiple domains, neural network architecture design and hyperparameter optimization have become crucial factors influencing model performance. Traditional hyperparameter optimization (HPO) methods, such as grid search, random search, and Bayesian optimization, often incur substantial computational costs and exhibit limited generalization ability. Meta-learning, following the paradigm of “learning to learn,” enables the extraction of transferable knowledge from prior tasks and facilitates rapid adaptation to new ones, offering a promising pathway toward automated and adaptive hyperparameter optimization. This paper provides a systematic survey of meta-learning-based approaches for hyperparameter optimization in neural networks. It first outlines the research background and existing challenges of HPO, then analyzes the core mechanisms of meta-learning in this context, encompassing optimization-based, model-based, and metric-based methods. Furthermore, it explores how the integration of reinforcement learning (RL) and dimensionality reduction techniques can enhance the performance and efficiency of meta-learning optimizers. Based on a comparative analysis of recent meta-learning-driven HPO studies across different application scenarios, the paper summarizes current progress, identifies open challenges, and outlines promising future research directions. Finally, it synthesizes the limitations of existing work and envisions emerging trends in this rapidly evolving field.