Loss Function
How do you answer when you are asked, "How happy are you now?". We may usually answer vaguely: "I am moderately happy" or "I am not very happy." You may be surprised if someone answers, "My current happiness score is 10.23" because the person can only quantify their happiness with one score. If such a person exists, the person may lead their life only based on their "happiness score."
This "happiness score" is an allegory used to illustrate some similar things which occur in neural network training. In neural network training, one "score" is used to indicate the current status. Based on the score, optimal weight parameters are searched for. As this person looks for an "optimal life" based on the "happiness score," a neural network searches for optimal parameters using "one score" as a guide. The score that's used in neural network training is called a loss function...