Machine Learning Model Fundamentals

Machine learning models are mathematical tools that allow us to uncover synthetic representations of external events, with the purpose of gaining better understanding and predicting future behavior. Sometimes these models have only been defined from a theoretical viewpoint, but advances in research now allow us to apply machine learning concepts to better understand the behavior of complex systems such as deep neural networks. In this chapter, we're going to introduce and discuss some fundamental elements. Skilled readers may already know these elements, but here we offer several possible interpretations and applications.

In particular, in this chapter, we're discussing the main elements of:

• Defining models and data
• Understanding the structure and properties of good datasets
• Scaling datasets, including scalar and robust scaling
• Normalization and whitening
• Selecting training, validation and test sets, including cross-validation
• The features of a machine learning model
• Learnability
• Capacity, including Vapnik-Chervonenkis capacity
• Bias, including underfitting
• Variance, including overfitting and the Cramér-Rao bound