Nature-based algorithms for deep learning based systems and applications

Deep Learning Based Systems (DLBS), characterized by their layered processing, in-model feature transformation, and high complexity, have revolutionized problem-solving across numerous domains. However, the manual design of optimal DLBS architectures is prohibitively time-consuming and resource-intensive. Nature-Based Algorithms (NBA), inspired by natural and biological processes, present a promising solution for automating this optimisation due to their ability to handle non-differentiable, discontinuous, and multi-modal problems. This research systematically addresses key challenges in applying NBA to optimise DLBS across distinct problem types. First, addressing the optimisation of complex DLBS for tabular data classification, we developed the MUlti-Layer heterogeneous Ensemble System (MULES) and the COnnection framework for Multi-layer Ensemble (COME). MULES introduces a novel NBA approach using NSGA-II to simultaneously select optimal classifiers and features at each layer of a DLBS. COME pioneers an NBA-driven framework to discover optimal inter-layer connections between classifiers within DLBS, moving beyond fixed input structures for subsequent layers. Turning to the critical challenge of computational expense in NBA for DLBS, particularly with variable-length architectural encodings, we proposed a novel LSTM-based encoder-decoder surrogate model integrated within a Surrogate-Assisted Evolutionary Algorithm (SAEA) framework. This innovation effectively bridges the gap between the variable-length encodings essential for representing diverse CNN architectures and the fixed-length input requirements of standard surrogate models. Finally, focusing on the application of NBA-optimised DLBS for medical image segmentation, we made two major contributions. We developed a weighted ensemble framework where optimal weights for combining predictions from diverse deep segmentation models are determined efficiently using NBA, maximising the Dice coefficient. Furthermore, we introduced a surrogate-based Optimal Decision Template method (ODTwS) that leverages surrogate models to drastically reduce the cost of optimising decision templates. The algorithms and frameworks developed throughout this thesis will be made publicly available, which would provide robust tools to support both academia and industry.