The bootstrap in regression analysis

We saw already in Chapter 7, Resampling Methods for estimation of the variance of MCD-based standard errors of correlation coefficients, resampling methods might be the only choice for estimating the variance for complex estimators. This is also true for regression analysis as soon as the classical, ordinary least-squares (OLS) regression is - for good reasons - skipped, and more robust methods are chosen.

Motivation to use the bootstrap

One might ask; "Why do we need a bootstrap to estimate the variance of regression coefficients when analytical expressions are known for it?". The answer is simple: because only for the ordinary least-squares regression, in addition to many model assumptions, are the analytical expressions valid.

Let's first look at the choice of more complex regression methods on a simple example using artificial data that best shows the problem that frequently occurs in practice:

library("robustbase")
data("hbk")
## structure of the data...