Understanding artificial neural networks
Deep learning utilizes artificial neural networks that consist of connected neurons or nodes. The following diagram represents a shallow neural network (single layer) to highlight the different components:
Figure 13.1 – A single-layer neural network
A network with more than one hidden layer is considered a deep neural network. In Figure 13.1, there are three layers – an input layer, a hidden layer, and an output layer.
The hidden layer represents a layer of connected neurons that perform a mathematical function. In its basic form, a neuron performs a linear function. For example, the first neuron in the hidden layer will perform a simple linear transformation – 
. Improving how neurons pass information from one layer to another is done by adding an activation function. For example, common activation functions for the hidden layer nodes include Sigmoid, ReLU, or Tanh, which are non-linear functions...
