The essence of regression

The first linear regression model was built by sir Francis Galton in 1908. The word regression implies towards the center. The covariates, also known as independent variables, features, or regressors, have a regressive effect on the output, also called dependent or regressand variable. Since the covariates are allowed and assumed to affect the output in linear increments, we call the model the linear regression model. The linear regression models provide an answer for the correlation between the regressand and the regressors and, as such, do not really establish causation.

As will be seen later in the chapter, using data, we will be able to understand the mileage of a car as a linear function of the car-related dynamics. From a purely scientific point of view, the mileage should really depend on complicated formulas of the car's speed, road conditions, climate, and so on.

However, it will be seen that linear models work just fine for the problem despite not really...