Sketching our roadmap

Dimensionality reduction can be roughly grouped into feature selection and feature projection methods. We have already employed some kind of feature selection in almost every chapter so far when we invented, analyzed, and then probably dropped some features. In this chapter, we will present some ways that use statistical methods—namely correlation and mutual information—to be able to do so in vast feature spaces. Feature projection tries to transform the original feature space into a lower-dimensional feature space. This is especially useful when we cannot get rid of features using selection methods, but we still have too many features for our learner. We will demonstrate this using principal component analysis (PCA), linear discriminant analysis (LDA), and multidimensional scaling (MDS).