Neural Network Conversion of Machine Learning Pipelines

Read full text Summarise with VIRAT
Authors: Man-Ling Sung, Jan Silovský, Man-Hung Siu, H. Gish, Chinnu Pittapally
Cited by: 0
References: 7

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Transfer learning and knowledge distillation has recently gained a lot of attention in the deep learning community. One transfer approach, the student-teacher learning, has been shown to successfully create ``small'' student neural networks that mimic the performance of a much bigger and more complex ``teacher'' networks. In this paper, we investigate an extension to this approach and transfer from a non-neural-based machine learning pipeline as teacher to a neural network (NN) student, which would allow for joint optimization of the various pipeline components and a single unified inference engine for multiple ML tasks. In particular, we explore replacing the random forest classifier by transfer learning to a student NN. We experimented with various NN topologies on 100 OpenML tasks in which random forest has been one of the best solutions. Our results show that for the majority of the tasks, the student NN can indeed mimic the teacher if one can select the right NN hyper-parameters. We also investigated the use of random forest for selecting the right NN hyper-parameters.

Fields of Study & Subjects

Machine Learning and Data Classification Domain Adaptation and Few-Shot Learning Advanced Neural Network Applications

Publication details

Journal / source
arXiv (Cornell University)
Publication date
26 March 2026
Language
EN
Document type
Preprint