Neural networks are a particular kind of algorithm because they need to be trained, that is, their parameters need to be optimized for a specific task by making them learn from available data. Once the networks are optimized to perform well on this training dataset, they can be used on new, similar data to provide satisfying results (if the training was done properly).
Before solving the problem of our MNIST task, we will provide some theoretical background, cover different learning strategies, and present how training is actually done. Then, we will directly apply some of these notions to our example so that our simple network finally learns how to solve the recognition task!
