A NN is a function (let's denote it with f) that tries to approximate another target function, g. We can describe this relationship with the following equation:
Here, x is the input data and θ are the NN parameters (weights). The goal is to find such θ parameters with the best approximate, g. This generic definition applies for both regression (approximating the exact value of g) and classification (assigning the input to one of multiple possible classes) tasks. Alternatively, the NN function can be denoted as 
.
We'll start our discussion from the smallest building block of the NN—the neuron.
