Feature Reduction
Feature reduction helps get rid of redundant variables that reduce the model efficiency in the following ways:
Time to develop/train the model increases.
Interpretation of the results becomes tedious.
It inflates the variance of the estimates.
In this section, we will see three feature reduction techniques that help in improving the model efficiency.
Principal Component Analysis (PCA)
N. A. Campbell and William R. Atchley in their classic paper, The Geometry of Canonical Variate Analysis, Systematic Biology, Volume 30, Issue 3, September 1981, Pages 268–280, geometrically defined a principal component analysis as a rotation of the axes of the original variable coordinate system to new orthogonal axes, called principal axes, such that the new axes coincide with directions of maximum variation of the original observation. This forms the crux of what PCA does. In other words, it represents the original variable with principal components that explain the maximum variation of the...