The rules of probability
At the simplest level, a model, be it machine learning or a more classical method such as linear regression, is a mathematical description of how a target variable changes in response to variation in a predictive variable; that relationship could be a linear slope or any of a number of more complex mathematical transformations. In the task of modeling, we usually think of separating the variables in our dataset into two broad classes:
- Independent data, by which we primarily mean inputs to a model, is often denoted by X. For example, if we are trying to predict the grades of school students on an end-of-year exam based on their characteristics, we could think of several kinds of features:
- Categorical: If there are six schools in a district, the school that a student attends could be represented by a six-element vector for each student. The elements are all zero, except for one which is “1,” indicating which of the six schools they are enrolled in...
