Random Forests
Chapter 3, Bagging, generalized the decision tree using the bootstrap principle. Before we embark on a journey with random forests, we will quickly review the history of decision trees and highlight some of their advantages and drawbacks. The invention of decision trees followed through a culmination of papers, and the current form of the trees can be found in detail in Breiman, et al. (1984). Breiman's method is popularly known as Classification and Regression Trees, aka CART. Around the late 1970s and early 1980s, Quinlan invented an algorithm called C4.5 independently of Breiman. For more information, see Quinlan (1984). To a large extent, the current form of decision trees, bagging, and random forests is owed to Breiman. A somewhat similar approach is also available in an algorithm popularly known by the abbreviation CHAID, which stands for Chi-square Automatic Interaction Detector. An in-depth look at CART can be found in Hastie, et al. (2009), and a statistical...
