Training a Neural Network

After network architecture and activation functions, the last design step before you can start training a neural network is the choice of loss function.

We will start with an overview of possible loss functions for regression, binary classification, and multiclass classification problems. Then, we will introduce some optimizers and additional training parameters for the training algorithms.

Loss Functions

In order to train a feedforward neural network, an appropriate error function, often called a loss function, and a matching last layer have to be selected. Let's start with an overview of commonly used loss functions for regression problems.

Loss Functions for Regression Problems

In the case of a regression problem, where the goal is to predict one single numerical value, the output layer should have one unit only and use the linear activation function. Possible loss functions to train this kind of network must refer to numerical error...