In this chapter, we answered a very important question: what does it mean for a model to work correctly? We explored the nature of errors and studied metrics that can quantify and measure model errors. We drew a line between offline and online model testing and defined testing procedures for both types. We can perform offline model testing using train/validation/test data splits and cross-validation. For online testing, we can choose between hypothesis tests and MABs.
In the next chapter, we will look into the inner workings of data science. We will dive into the main concepts behind machine learning and deep learning, giving an intuitive understanding of how machines learn.
