Activation Functions

An activation function formulates how the inputs to a particular neuron will be processed to generate an output.As shown in the following diagram, each of the neurons in a neural networkneural network has an activation function that determines how inputs will be processed:

Figure 8.9: Activation Function

In the preceding diagram, we can see that the results generated by an activation function are passed on to the output. The activation function sets the criteria that how the values of the inputs are supposed to be interpreted to generate an output.For exactly the same input values, different activation functions will produce different outputs. Understanding how to select the right activation function is important when using neural neural networks to solve problems.Let's now look into these activation functions one by one.