Loss functions – what are they?
A loss function takes two inputs; for example, a model prediction and the corresponding ground-truth value. It then compares the two inputs and summarizes this comparison into a single number.
Let’s take that example further. We’ll denote the ground-truth value by 
and the model prediction by 
. A loss function in this example would then be a function of both and , which returns a single real number. Let’s call that loss function 
. We’ll meet a concrete example of a loss function in the next section. But for now, it suffices to say that a loss function 
attempts to measure how similar 
is to , with a loss function value of zero indicating that ˆ y is identical to y.
In general, a lower value of the function means that is closer or more similar to 
, while a higher value of the loss function means that is further from or less similar to .
When training a model, our training data will...
