Restricted Boltzmann Machines (RBMs)

The RBM is a probabilistic, undirected, graphical model with an input layer of observable variables or features and a layer of latent, representative neurons. It is also interpreted as a stochastic neural network. RBMs can be stacked to compose a deep belief network (DBM) [11:3].

The simplest form of RBM architecture consists of one layer of input variables and a layer of latent variables.

Boltzmann machine

The Boltzmann machine is a symmetric network of binary vectors of stochastic processing nodes. It is an undirected structure, used to discover interesting features in datasets composed of binary vectors and the probability distribution of the input data. It is commonly associated with Markov random fields [11:4].

Note

For a neuron i, weights w_ij of connections from units j, and a bias unit b_i, an energy E(x_i) and input x_i, probability for unit i:

Note

Boltzmann probability

E(x=0) being the energy for negative outcome and E(x=1) being the energy for the...