Dictionary learning is a technique that allows you to rebuild a sample starting from a sparse dictionary of atoms (similar to principal components, but without constraints about the independence). Conventionally, when the dictionary contains a number of elements less than the dimensionality of the samples m, it is called under-complete, and on the other hand it's called over-complete when the number of atoms is larger (sometimes much larger) than m.
In Online Dictionary Learning for Sparse Coding, Mairal J., Bach F., Ponce J., Sapiro G., Proceedings of the 29th International Conference on Machine Learning, 2009, there's a description of the same online strategy adopted by scikit-learn, which can be summarized as a double optimization problem.
Let's suppose that we have a dataset, X:
Our goal is to find both a dictionary D...
