Restricted Boltzmann Machines

RBMs are a family of unsupervised feature learning algorithms that use probabilistic models to learn new features. Like PCA and LDA, we can use RBMs to extract a new feature set from raw data and use them to enhance machine learning pipelines. The features that are extracted by RBMs tend to work best when followed by linear models such as linear regression, logistic regression, perceptron's, and so on.

The unsupervised nature of RBMs is important as they are more similar to PCA algorithms than they are to LDA. They do not require a ground-truth label for data points to extract new features. This makes them useful in a wider variety of machine learning problems.

Conceptually, RBMs are shallow (two-layer) neural networks. They are thought to be the building blocks of a class of algorithms called Deep Belief Networks (DBN). Keeping with standard terminology, there is a visible layer (the first layer), followed by a hidden layer (the second layer). These are the only...