Hebb's rule

Hebb's rule has been proposed as a conjecture in 1949 by the Canadian psychologist Donald Hebb to describe the synaptic plasticity of natural neurons. A few years after its publication, this rule was confirmed by neurophysiological studies, and many research studies have shown its validity in many application, of Artificial Intelligence. Before introducing the rule, it's useful to describe the generic Hebbian neuron, as shown in the following diagram:

Generic Hebbian neuron with a vectorial input

The neuron is a simple computational unit that receives an input vector x, from the pre-synaptic units (other neurons or perceptive systems) and outputs a single scalar value, y. The internal structure of the neuron is represented by a weight vector, w, that models the strength of each synapse. For a single multi-dimensional input, the output is obtained as follows: 

In this model, we are assuming that each input signal is encoded in the corresponding component of the vector, x; therefore...