For this chapter, you will need to install the following software, if you haven't already done so:

• Jupyter Notebook
• Anaconda
• Python

The code files for this chapter can be found at https:/​/​github.​com/​PacktPublishing/
Supervised-Machine-Learning-with-Python.

When it comes to supervised learning, there are two families of learning algorithms: parametric and non-parametric. This area also happens to be a hotbed for gatekeeping and opinion-based conjecture regarding which is better. Basically, parametric models are finite-dimensional, which means that they can learn only a defined number of model parameters. Their learning stage is typically categorized by learning some vector theta, which is also called a coefficient. Finally, the learning function is often a known form, which we will clarify later in this section.

If we go back to our definition of supervised learning, recall that we need to learn some function, f. A parametric...

Linear regression solves the least squares equation to discover the parameters vector theta. In this section, we will walk through the source code for a linear regression class in the packtml Python library and then cover a brief graphical example in the examples directory.

Before we look at the code, we will be introduced to the interface that backs all of the estimators in the book. It is called BaseSimpleEstimator, which is an abstract class. It's going to enforce only one method, which is predict. Different subclass layers are going to enforce other methods for different model families. But this layer backs all the models that we will build, as everything that we are putting together is supervised, so it's all going to need to be able to predict. You will notice that the signature is prescribed in the dock string. Every...

In this section, we will look at logistic regression, which is the first hill-climbing algorithm that we'll cover, and we will have a brief recap of linear regression. We will also look at how logistic regression differs both mathematically and conceptually. Finally, we will learn the core algorithm and explain how it makes predictions.

Logistic regression is conceptually the inverse of linear regression. What if, rather than a real value, we want a discrete value or a class? We have already seen one example of this type of question early on when we wanted to predict whether or not an email was spam. So, with logistic regression, rather than predicting a real value, we can predict...

In this section, we will walk through the implementation of logistic regression in Python within the packtml package. We will start off with a brief recap of what logistic regression seeks to accomplish and then go over the source code and look at an example.

Notice that we have three Python files open. One is extmath.py, from within the utils directory inside of packtml; another is simple_logistic.py, from within the regression library in packtml; and the final one is an example_logistic_regression.py file, inside the examples directory and regression.

We will dive right into the code base...

Parametric models have some really convenient attributes. Namely, they are fast to fit, don't require too much data, and can be very easily explained. In the case of linear and logistic regression, it's easy to look at coefficients and directly explain the impact of fluctuating one variable in either direction. In regulated industries, such as finance or insurance, parametric models tend to reign supreme, since they can be easily explained to regulators. Business partners tend to really rely on the insights that the coefficients produce. However, as is evident in what we've already seen so far, they tend to oversimplify. So, as an example, the logistic regression decision boundary that we looked at in the last section assumes a perfect linear boundary between two classes.

It is rare that the real world can be constrained into...