The perceptron

Let's understand the most basic building block of a neural network, the perceptron, also known as the artificial neuron. The concept of the perceptron originated in the works of Frank Rosenblatt in 1962.

You may want to read the following work to explore the origins of neural networks:

Frank Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books, 1962

In the most simplified view, a perceptron is modeled after the biological neurons such that it takes one or multiple inputs and combines them to generate output.

As shown in the following image, the perceptron takes three inputs and adds them to generate output y:

Simple perceptron

This perceptron is too simple to be of any practical use. Hence, it has been enhanced by adding the concept of weights, bias, and activation function. The weights are added to each...