Adaptive Negative Correlation Learning via Online Hypergradient Descent: Toward Self-Regulating Diversity in Neural Network Ensembles

This thesis investigates two adaptive formulations of Negative Correlation Learning (NCL), an ensemble learning technique that explicitly regulates diversity among neural network learners to reduce redundant learning and improve generalization. Traditional NCL relies on a fixed diversity penalty coefficient , which balances individual model accuracy against ensemble diversity. However, selecting an appropriate typically requires an expensive hyperparameter search and remains misaligned with the inherently dynamic nature of training, where the optimal strength of diversity pressure shifts as learning progresses. To address this limitation, the present work introduces two novel adaptive NCL variants that replace the static penalty with learnable parameters updated via hypergradient descent. First, a global- formulation that learns a single shared coefficient for the entire ensemble. And, second, a multi- formulation where each learner maintains its own adaptive penalty. Both variants employ forward-mode differentiation to compute the hypergradient (the gradient of the validation loss with respect to ) which directly quantifies how changes in the diversity penalty are expected to affect generalization. Because forward-mode can be adapted to compute this quantity alongside the usual parameter updates, it enables real time adjustmentsto at every training step. Thus, the ensemble can continuously regulate its own diversity strength as training evolves. Experimental results across classification and regression tasks show that both adaptive methods achieve performance comparable to classical NCL and bagging. Ablation studies reveal that the learned coefficients evolve in stable and interpretable ways, correcting extreme initializations and responding to changes in the validation signal. Furthermore, the adaptive mechanisms consistently integrated well with the ensemble and demonstrated robustness to initialization and hypergradient settings. Overall, the findings indicate that treating the NCL penalty strength as a learnable quantity is feasible and provides a principled alternative to manual hyperparameter tuning. The code used to reproduce the experiments is available at: https://github.com/fmportela/adaptive-ncl-hgd.