Introduction to RBMs

RBMs are unsupervised models that can be used in different applications that require rich latent representations. They are usually used in a pipeline with a classification model with the purpose of extracting features from the data. They are based on Boltzmann Machines (BMs), which we discuss next (Hinton, G. E., and Sejnowski, T. J. (1983)).

BMs

A BM can be thought of as an undirected dense graph, as depicted in Figure 10.1:

Figure 10.1 – A BM model

This undirected graph has some neural units that are modeled to be visible, , and a set of neural units that are hidden, . Of course, there could be many more than these. But the point of this model is that all neurons are connected to each other: they all talk among themselves. The training of this model will not be covered here, but essentially it is an iterative process where the input is presented in the visible layers, and every neuron (one at a time) adjusts its connections with other neurons to satisfy...