Efficient neural network architecture search

Read full text Summarise with VIRAT
Authors: Roxana Istrate
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

In recent years an increasing number of researchers and practitioners focused on algorithms for large-scale neural network architecture search. Consequently, numerous publications started to push the boundaries of accuracy, however at immense training costs. For tens of days, hundreds of GPUs evaluate thousands of candidates in order to discover a configuration that surpasses the accuracy of hand designed networks or of the results of other automated searches. After designing our own type of search and analysing its drawbacks, we observed repetitive evaluations of similar candidates. Unless the intermediate results are wisely reused, architecture searches started from scratch have high chances of wasting compute time on similar or identical candidates. We propose to speed up the evaluation phase by replacing the slow training process with a fast and reliable prediction of the candidate’s performance. In contrast to previously proposed approaches, our prediction is not only calibrated on the topological network information, but also on the characterization of the dataset-difficulty which allows us to re-tune the prediction for unseen datasets without any training. We employ our predictor in image and text classification tasks, and emphasize oil the delivered speed-up in comparison with related work.

Fields of Study & Subjects

Machine Learning and Data Classification Advanced Neural Network Applications Domain Adaptation and Few-Shot Learning

Publication details

Journal / source
Research Portal (Queen's University Belfast)
Publication date
1 October 2026
Language
EN
Document type
Thesis