Trees and forests

In this chapter tree based classification models are considered. The chapter starts with an introduction to classification trees, and an explanation of how visualisation of the data can help us understand if a tree model is appropriate. It then gives a brief introduction into the random forest algorithm and the particular outputs for this class of models that can be used for interpretation. First, it is demonstrated how the votes matrix can be examined in a simplex for any number of classes in the data. By examining the distribution of points betweem vertices, we can explain the extent to which classes are more often confused, and which observations the model confuses between multiple classes. Finally, the chapter also shows how to use variable importance measures to select variables to visualise using a tour, for understanding the overall model, and using the class-wise importance measures to understand class differences. Using the tour can also help to see where these particular variable importance metrics do less well to capture the multivariate nature of class separations.