Dimensionality Reduction with PCA
Dimensionality reduction can be as simple as removing unimportant features from the training data. However, it's usually not obvious that removing a set of features will boost model performance. Even features that are highly noisy may offer some valuable information that models can learn from. For these reasons, we should know about better methods for reducing data dimensionality, such as the following:
- Principal Component Analysis (PCA)
- Linear Discriminant Analysis (LDA)
These techniques allow for data compression, where the most important information from a large group of features can be encoded in just a few features.
In this section, we'll focus on PCA. This technique transforms the data by projecting it into a new subspace of orthogonal principal components, where the components with the highest eigenvalues (as described here) encode the most information for training the model. Then, we can simply select a set of...
