Assessing Model Accuracy Using Cross-Validation

This chapter focuses on assessing model accuracy using cross-validation techniques. In addition to the Summary, there are six sections in Chapter 5 . Section 5.1 introduces four types of model performance metrics in a regression setting: root mean squared error (RMSE), R-squared ( https://www.w3.org/1998/Math/MathML" display="inline"> R 2 ), adjusted https://www.w3.org/1998/Math/MathML" display="inline"> R 2 , and mean absolute error (MAE). Section 5.2 introduces five cross-validation techniques: (1) hold-out set method, (2) leave-one-out cross-validation, (3) k -fold cross-validation, (4) repeated k -fold cross-validation, and (5) random cross-validation. Section 5.3 introduces an analysis of model accuracy using cross-validation. Section 5.4 performs model selection using repeated cross-validation. Section 5.5 checks whether cross-validation or original data provide more accurate model prediction assessed by https://www.w3.org/1998/Math/MathML" display="inline"> R 2 . Section 5.6 provides some remarks on cross-validation and data-splitting methods.