Understanding bagging
Bagging is an abbreviation for bootstrap aggregation. The important underlying concept here is the bootstrap, which was invented by the eminent scientist Bradley Efron. We will first digress here slightly from the CART technique and consider a very brief illustration of the bootstrap technique.
The bootstrap
Consider a random sample 
of size n from 
. Let 
be an estimator of 
. To begin with, we first draw a random sample of size n from with a replacement; that is, we obtain a random sample 
, where some of the observations from the original sample may have repetitions and some may not be present at all. There is no one-to-one correspondence between and . Using , we compute 
. Repeat this exercise several times, say B. The inference for is carried out by using the sampling distribution of the bootstrap samples , …, 
.
Let us illustrate the concept of the bootstrap with the famous aspirin example; see Chapter 8 of Tattar, et. al. (2013). A surprising double-blind experiment...
