Loss functions
Loss function is a necessary part, because it is what we want to minimize during the training. You can find a few popular loss functions in the table:
Name | Formula | Usually used for |
Mean squared error or L2-loss | Regression | |
Mean absolute error or L1-loss | Regression | |
Categorical cross entropy | Softmax multiclass classification |
Where y is a ground-truth vector and ŷ is a vector of predictions of length n.