We have previously seen, in Chapter 5, Training a Single Neuron, that Rosenblatt's perceptron model is simple and powerful for some problems (Rosenblatt, F. 1958). However, for more complicated and highly non-linear problems, Rosenblatt did not give enough attention to his models that connected many more neurons in different architectures, including deeper models (Tappert, C. 2019).
Years later, in the 1990s, Prof. Geoffrey Hinton, the 2019 Turing Award winner, continued working to connect more neurons together since this is more brain-like than simple neurons (Hinton, G. 1990). Most people today know this type of approach as connectionist. The main idea is to connect neurons in different ways that will resemble brain connections. One of the first successful models was the MLP, which uses a supervised gradient descent-based learning algorithm that learns to approximate a function, 
, using labeled data, 
.
Figure 6.1 depicts an MLP with one layer of multiple neurons...
