The relation to statistical learning theory

We have seen a brief history of how AI has developed and the modern rise of ML as the most promising approach to many applications. Apparently, the technology shift isn’t an isolated event; it is being realized along with the advance of computing technologies, such as high-performance computing and big data technology. The mathematical foundation of ML algorithms utilizes a heavy implementation of modern statistics. For a long time, statisticians thought computer scientists were reinventing the wheel, while computer scientists were seeking more practical realization of statistics-driven methods. The truth is that statistics is studying the properties of data generation processes. However, ML is a relatively new paradigm that is quite the opposite: that is, it studies the data generation process given some observations. In other words, they are closely related and complementary to each other. In Figure 4.2, we relate statistics and...