Refining the training process

Network architectures are not the only things to have improved over the years. The way that networks are trained has also evolved, improving how reliably and quickly they can converge. In this section, we will tackle some of the shortcomings of the gradient descent algorithm we covered in Chapter 1, Computer Vision and Neural Networks, as well as some ways to avoid overfitting.