To work on the recipes in this chapter, you need the following files (available on GitHub):

• simple_classifier.py
• logistic_regression.py
• naive_bayes.py
• data_multivar.txt
• splitting_dataset.py
• confusion_matrix.py
• performance_report.py
• car.py
• car.data.txt
• income.py
• adult.data.txt
• wine.quality.py
• wine.txt
• post.classification

In the field of machine learning, classification refers to the process of using the characteristics of data to separate it into a certain number of classes. This is different than regression, which we discussed in Chapter 1, The Realm of Supervised Learning, where the output is a real number. A supervised learning classifier builds a model using labeled training data and then uses this model to classify unknown data.

A classifier can be any algorithm that implements classification. In simple cases, a classifier can be a straightforward mathematical function. In more real-world cases, a classifier can take very complex forms. In the course of study, we will see that classification can be either binary, where we separate data into two classes, or it can be multi-class, where we separate data into more than two classes. The mathematical techniques that are devised to...

A classifier is a system with some characteristics that allow you to identify the class of the sample examined. In different classification methods, groups are called classes. The goal of a classifier is to establish the classification criterion to maximize performance. The performance of a classifier is measured by evaluating the capacity for generalization. Generalization means attributing the correct class to each new experimental observation. The way in which these classes are identified discriminates between the different methods that are available.

Classifiers identify the class of a new objective, based on knowledge that's been extracted from a series of samples (a dataset...

Despite the word regression being present in the name, logistic regression is actually used for classification purposes. Given a set of datapoints, our goal is to build a model that can draw linear boundaries between our classes. It extracts these boundaries by solving a set of equations derived from the training data. In this recipe, we will build a logistic regression classifier.

Logistic regression is a non-linear regression model used when the dependent variable is dichotomous. The purpose is to establish the probability with which an observation can generate one or the other value of the dependent variable; it can also be used to classify observations, according...

A classifier solves the problem of identifying sub-populations of individuals with certain features in a larger set, with the possible use of a subset of individuals known as a priori (a training set). A Naive Bayes classifier is a supervised learning classifier that uses Bayes' theorem to build the model. In this recipe, we will build a Naive Bayes classifier.

The underlying principle of a Bayesian classifier is that some individuals belong to a class of interest with a given probability based on some observations. This probability is based on the assumption that the characteristics observed can be either dependent or independent from one another; in this second case...

Let's see how to split our data properly into training and testing datasets. As we said in Chapter 1, The Realm of Supervised Learning, in the Building a linear regressor recipe, when we build a machine learning model, we need a way to validate our model to check whether it is performing at a satisfactory level. To do this, we need to separate our data into two groups—a training dataset and a testing dataset. The training dataset will be used to build the model, and the testing dataset will be used to see how this trained model performs on unknown data.

In this recipe, we will learn how to split the dataset for training and testing phases.

The fundamental...

Cross-validation is an important concept in machine learning. In the previous recipe, we split the data into training and testing datasets. However, in order to make it more robust, we need to repeat this process with different subsets. If we just fine-tune it for a particular subset, we may end up overfitting the model. Overfitting refers to a situation where we fine-tune a model to a dataset too much and it fails to perform well on unknown data. We want our machine learning model to perform well on unknown data. In this recipe, we will learn how to evaluate model accuracy using cross-validation metrics.

When we are dealing with machine learning models, we...

A confusion matrix is a table that we use to understand the performance of a classification model. This helps us understand how we classify testing data into different classes. When we want to fine-tune our algorithms, we need to understand how data gets misclassified before we make these changes. Some classes are worse than others, and the confusion matrix will help us understand this. Let's look at the following:

In the preceding diagram, we can see how we categorize data into different classes. Ideally, we want all the non-diagonal elements to be 0. This would indicate perfect classification! Let's consider class 0. Overall, 52 items actually belong to class 0. We get 52 if we sum up the numbers in the first row. Now, 45 of these items are being predicted correctly, but our classifier says that 4 of them belong to class 1 and three...

In the Evaluating accuracy using cross-validation metrics recipe, we calculated some metrics to measure the accuracy of the model. Let's remember its meaning. The accuracy returns the percentage of correct classifications. Precision returns the percentage of positive classifications that are correct. Recall (sensitivity) returns the percentage of positive elements of the testing set that have been classified as positive. Finally, in F1, both the precision and the recall are used to compute the score. In this recipe, we will learn how to extract a performance report.

We also have a function in scikit-learn that can directly print the precision, recall, and F1 scores for us...

In this recipe, let's see how we can apply classification techniques to a real-world problem. We will use a dataset that contains some details about cars, such as number of doors, boot space, maintenance costs, and so on. Our goal is to determine the quality of the car. For the purposes of classification, quality can take four values: unacceptable, acceptable, good, or very good.

You can download the dataset at https://archive.ics.uci.edu/ml/datasets/Car+Evaluation.

You need to treat each value in the dataset as a string. We consider six attributes in the dataset. Here are the attributes along with the possible values they can take:

• buying: These will be vhigh...

We used random forests to build a classifier in the previous recipe, Evaluating cars based on their characteristics, but we don't exactly know how to define the parameters. In our case, we dealt with two parameters: n_estimators and max_depth. They are called hyperparameters, and the performance of the classifier depends on them. It would be nice to see how the performance gets affected as we change the hyperparameters. This is where validation curves come into the picture.

Validation curves help us understand how each hyperparameter influences the training score. Basically, all other parameters are kept constant and we vary the hyperparameter of interest according to our range...

Learning curves help us understand how the size of our training dataset influences the machine learning model. This is very useful when you have to deal with computational constraints. Let's go ahead and plot learning curves by varying the size of our training dataset.

A learning curve shows the validation and training score of an estimator for varying numbers of training samples.

Let's see how to extract learning curves:

1. Add the following code to the same Python file as in the previous recipe, Extracting...

We will build a classifier to estimate the income bracket of a person based on 14 attributes. The possible output classes are higher than 50,000 or lower than or equal to 50,000. There is a slight twist in this dataset, in the sense that each datapoint is a mixture of numbers and strings. Numerical data is valuable, and we cannot use a label encoder in these situations. We need to design a system that can deal with numerical and non-numerical data at the same time.

We will use the census income dataset available at https://archive.ics.uci.edu/ml/datasets/Census+Income.

The dataset has the following characteristics:

• Number of instances: 48,842
• Number of attributes: 14

The following...

In this recipe, we will predict the quality of wine based on the chemical properties of wines grown. The code uses a wine dataset, which contains a DataFrame with 177 rows and 13 columns; the first column contains the class labels. This data is obtained from the chemical analyses of wines grown in the same region in Italy (Piemonte) but derived from three different cultivars—namely, the Nebbiolo, Barberas, and Grignolino grapes. The wine from the Nebbiolo grape is called Barolo.

The data consists of the amounts of several constituents found in each of the three types of wines, as well as some spectroscopic variables. The attributes are as follows:

• Alcohol
• Malic acid
...

Newsgroups are discussion groups on many issues and are made available by news-servers, located all over the world, which collect messages from clients and transmit them, on the one hand, to all their users and, on the other, to other news-servers connected to the network. The success of this technology is due to user interaction in discussions. Everyone has to respect the rules of the group.

In this recipe, we will build a classifier that will allow us to classify the membership of a topic into a particular discussion group. This operation will be useful to verify whether the topic is relevant to the discussion group. We will use the data contained in the 20 newsgroups...