Overfitting and Underfitting

This chapter introduces two pairs of basic concepts in machine learning: model complexity and interpretability, and overfitting and underfitting. In addition to the Summary, there are five sections. Section 4.1 introduces model complexity and interpretability. Section 4.2 defines overfitting and underfitting. Section 4.3 introduces how to detect overfitting and underfitting, including using visualization, checking accuracy, and k-fold cross-validation techniques. Section 4.4 introduces how to deal with overfitting and underfitting, including bias (underfitting) and variance (overfitting) tradeoff, L 1 and L 2 regularizations, ensemble learning, early stopping and dropout, training with more data, and data augmentation. Section 4.5 provides some remarks on overfitting and underfitting, including model selection and evaluation, optimization and loss function, and hyperparameter optimization.