Summary
This chapter described neural network training. First, we introduced a score called a loss function so that a neural network can learn. The goal of neural network training is to discover the weight parameters that lead to the smallest value of the loss function. Then, we learned how to use the gradient of a function, called the gradient method, to discover the smallest loss function value. This chapter covered the following points:
- In machine learning, we use training data and test data.
- Training data is used for training, while test data is used to evaluate the generalization capability of the trained model.
- A loss function is used as a score in neural network training. Weight parameters are updated so that the value of the loss function will decrease.
- To update the weight parameters, their gradients are used to update their values in the gradient direction repeatedly.
- Calculating a derivative based on the difference when very small values are provided...
