Bagging

Bootstrap aggregating, or bagging, is an ensemble meta-algorithm that can reduce the variance in an estimator. Bagging can be used in classification and regression tasks. When the component estimators are regressors, the ensemble averages their predictions. When the component estimators are classifiers, the ensemble returns the mode class.

Bagging independently fits multiple models on variants of the training data. The training data variants are created using a procedure called bootstrap resampling. Often it is necessary to estimate a parameter of an unknown probability distribution using only a sample of the distribution. We can use this sample to calculate a statistic, but we know that this statistic will vary according to the sample we happened to draw. Bootstrap resampling is a method of estimating the uncertainty in a statistic. It can only be used if the observations in the sample are drawn independently. Bootstrap resampling produces multiple variants of the sample by repeatedly...