Evaluating the Accuracy of a Regression Model

In order to evaluate regression models, we first need to define some metrics. The common metrics used to evaluate regression models rely on the concepts of residuals and errors, which are quantifications of how much a model mispredicts a particular data point. In the following sections, we will first learn about residuals and errors. We will then learn about two evaluation metrics, mean absolute error (MAE) and root mean squared error (RMSE), and how they are used to evaluate regression models.

Residuals and Errors

An important concept in understanding how to evaluate regression models is the residual. The residual refers to the difference between the value predicted by the model and the true value for a data point. It can be thought of as by how much your model missed a particular value. The following diagram illustrates this:

Figure 6.1: Estimating the residual

The residual is taken to be an estimate of the error of a model, where the error is...