Online learning in open feature spaces : exploring decision tree-based methods for learning from data streams with varying feature spaces

In an era of rapid technological advancements, the ability to efficiently understand and extract knowledge from dynamic and ever-evolving data environments is crucial. This thesis presents machine learning methods to deal with such intricacies by leveraging the foundations of decision trees and random forests. Specifically, the proposed approaches address the issues of monotonically increasing feature spaces and varying feature spaces in online learning environments. Beyond that, we argue that our proposed methods naturally convey a higher level of interpretability over their machine learning competitors tailored for the same evolving environments. The proposed methods are built on the foundation of decision tree-based models, which are inherently known for their interpretability. On the other hand, decision tree-based ensemble models are also particularly known for their performance. Hence, we envisioned adjusting and developing models in this group to provide performant and interpretable models that are capable of taking on the challenges that come with dynamic and ever-evolving environments. The first method, Dynamic Fast Decision Tree, is targeted at monotonically increasing feature spaces, i.e., feature spaces where new features emerge over time. The proposed approach extends the established Hoeffding tree method, with additional restructuring and pruning capabilities. The second approach, Dynamic Forest, is targeted at varying feature spaces, i.e., feature spaces where new features may emerge, while others may vanish. It builds on the well-established principles of random forest and enhances them by dynamically managing the ensemble to deal with the implications of varying feature spaces. The final method presented in this thesis is called ORF3V, which also focuses on varying feature spaces. The method dynamically manages so-called feature forests and forms a random forest-like ensemble, based on efficient approximation of feature statistics. Empirical evaluations were conducted for all three methods, demonstrating that the proposed methods achieve competitive results while having the advantage of being interpretable. Finally, we provide a roadmap for future directions and open challenges, which were identified in our previous work.