In this chapter, we will introduce and discuss some very important techniques that can be employed to perform both dimensionality reduction and component extraction. In the former case, the goal is to transform a high-dimensional dataset into a lower-dimensional one, to try to minimize the amount of information loss. The latter is a process that's needed to find a dictionary of atoms that can be mixed up, in order to build samples.
In particular, we will discuss the following topics:
- Principal Component Analysis (PCA)
- Singular Value Decomposition (SVD) and whitening
- Kernel PCA
- Sparse PCA and dictionary learning
- Factor analysis
- Independent Component Analysis (ICA)
- Non-Negative Matrix Factorization (NNMF)
- Latent Dirichlet Allocation (LDA)
