Designing your Network
In the previous section, we learned that neural networks are characterized by a topology, weights, and activation functions. In particular, feedforward neural networks have an input and an output layer, plus a certain number of hidden layers in between. While the values for the network weights are automatically estimated via the training procedure, the network topology and the activation functions have to be predetermined during network design before training. Different network architectures and different activation functions implement different input-output tasks. Designing the appropriate neural architecture for a given task is still an active research field in the deep learning area (Goodfellow I., Bengio Y., Courville A. (2016). Deep Learning, MIT Press).
Other parameters are involved in the training algorithm of neural networks, such as the learning rate or the loss function. We have also seen that neural networks are prone to overfitting; this means...
