Activation functions as hyperparameters

Activation functions, which are less commonly known as transfer functions, are used to enable the model to learn nonlinear prediction boundaries. Different activation functions behave differently and are carefully chosen based on the deep learning task at hand. We have already discussed different types of activation in an earlier section of this chapter, Understanding activation functions.

In the next section, we will learn about the popular deep learning APIs—TensorFlow and Keras.