Deep neural networks

Neural networks are extremely popular today, thanks to major research advancement over the last 10 years. The result of this research has culminated in deep learning algorithms and architecture. Big technology giants such as Google, Facebook, and Microsoft are heavily investing in deep learning network research. Complex neural networks powered by deep learning are considered state of the art in AI and machine learning. We see them being used in everyday life. For example, Google's image search is powered by deep learning. Google Translate is another application powered by deep learning today. The field of computer vision has made several advancements thanks to deep learning.

The following diagram is a typical neural network, commonly called a multi-layer perceptron:

This network architecture has a single hidden layer with two nodes. The output layer is activated by a softmax function. This network is built for a classification task. The hidden layer can be activated by...