Feed-forward neural networks

The idea behind artificial neural networks was to build mathematical and computational models of the natural neural network in the brain. After all, the brain is a very powerful information processing engine that surpasses computers in domains, such as learning, inductive reasoning, prediction and vision, and speech recognition.

The biological background

In biology, a neural network is composed of groups of neurons interconnected through synapses [9:1], as shown in the following diagram:

The visualization of biological neurons and synapses

Neuroscientists have been especially interested in understanding how billions of neurons in the brain can interact to provide human beings with parallel processing capabilities. The 60s saw a new field of study emerging, known as connectionism. Connectionism marries cognitive psychology, artificial intelligence, and neuroscience. The goal was to create a model for mental phenomena. Although there are many forms of connectionism...