Dimensionality reduction
One last subject we'll examine is dimensionality reduction. Another way to combat the curse of dimensionality is to reduce the size of our dimensions with mathematical techniques. There are several:
- Principle Component Analysis (PCA)
- Singular Value Decomposition (SVD, sometimes also called LSA)
- Independent Component Analysis (ICA)
- Non-Negative Matrix Factorization (NMF)
- t-SNE and UMAP (better for non-linear structures, like natural language and images)
- Autoencoders (a neural network technique that ends up being similar to PCA)
Covering all these methods is beyond the scope of this book, and we will only explore PCA. However, many other ML and feature engineering books cover these methods, such as Building Machine Learning Systems with Python – Third Edition, by Luis Pedro Coelho, Willi Richert, and Matthieu Brucher from Packt (which has an entire chapter on dimensionality reduction). Let's...
