Neural Networks and Deep Learning – Basic Concepts

All you hear about at the moment is deep learning. Deep learning stems from the traditional discipline of neural networks, in the realm of machine learning.

The field of neural networks has gone through a number of stop-and-go phases. Since the early excitement for the first perceptron in the '60s and the subsequent lull when it became evident what the perceptron could not do; through the renewed enthusiasm for the backpropagation algorithm applied to multilayer feedforward neural networks and the subsequent lull when it became apparent that training recurrent networks required hardware capabilities that were not available at the time; right up to today's new deep learning paradigms, units, and architectures running on much more powerful, possibly GPU-equipped, hardware.

Let's start from the beginning and, in this section, go through the basic concepts behind neural networks and deep learning. While these...