When working with standard PCA (or other techniques, such as factor analysis), the components are uncorrelated, but it's not guaranteed that they are statistically independent. In other words, let's suppose that we have a dataset, X, drawn from a joint probability distribution, p(X); if there are n components, we cannot always be sure that the following equality holds:
However, there are many important tasks, based on a common model called the cocktail party. In such scenarios, we can suppose (or we know) that many different and independent sources (for example, voices and music) overlap and generate a single signal. At this point, our goal is to try to separate the sources by applying a linear transformation to each sample. Let's consider a whitened dataset, X (so all of the components have the same informative content), which we...
