Fundamentals of Artificial Neural Networks

Given that autoencoders are based on artificial neural networks, an understanding of neural networks is also critical for understanding autoencoders. This section of the chapter will briefly review the fundamentals of artificial neural networks. It is important to note that there are many aspects of neural networks that are outside the scope of this book. The topic of neural networks could easily fill, and has filled, many books on its own, and this section is not to be considered an exhaustive discussion of the topic.

As described earlier, artificial neural networks are primarily used in supervised learning problems, where we have a set of input information, say a series of images, and we are training an algorithm to map the information to a desired output, such as a class or category. Consider the CIFAR-10 dataset shown in Figure 5.3 as an example, which contains images of 10 different categories (airplane, automobile, bird, cat, deer...