So far, we have used the Ordinary Least Squares (OLS) estimates for our linear regression models. But these models only become valid when all regression hypotheses are verified. If this is not the case, least squares regression can be problematic. In such cases we can try to locate the problems through residual diagnostics, but this procedure may be slow and requires a great deal of experience. Often, model-fitting problems are due to the presence of extreme values called outliers. The following figure shows a distribution with outliers:
Outliers have a large influence on the fit, because squaring the residuals magnifies the effects of these extreme data points. Outliers tend to change the direction of the regression line by getting much more weight than they are worth. Thus, the estimate of the regression coefficients is clearly distorted...
