Linear Regression

We will start our investigation into regression problems with the selection of a linear model. Linear models, while being a great first choice due to their intuitive nature, are also very powerful in their predictive power, assuming datasets contain some degree of linear or polynomial relationship between the input features and values. The intuitive nature of linear models often arises from the ability to view data as plotted on a graph and observe a trending pattern in the data with, say, the output (the y axis value for the data) trending positively or negatively with the input (x axis value). While often not presented as such, the fundamental components of linear regression models are also often learned during high school mathematics classes. You may recall that the equation of a straight line, or linear model, is defined as follows:

Figure 3.1: Equation of a straight line

Here, x is the input value and y is the corresponding output or predicted value. The parameters of...