In this chapter, we will use the Matplotlib, pandas, NumPy, scikit-learn, feature-engine, and Category Encoders Python libraries. If you need to install Python, the free Anaconda Python distribution (https://www.anaconda.com/) includes most numerical computing libraries.

feature-engine can be installed with pip:

pip install feature-engine

If you use Anaconda, you can install feature-engine with conda:

conda install -c conda-forge feature_engine

To install Category Encoders, use pip as follows:

pip install category_encoders

We will use the Credit Approval dataset from the UCI Machine Learning Repository (https://archive.ics.uci.edu/), licensed under the CC BY 4.0 creative commons attribution: https://creativecommons.org/licenses/by/4.0/legalcode. You’ll find the dataset at this link: http://archive.ics.uci.edu/dataset/27/credit+approval.

I downloaded and modified the data as shown in this notebook: https://github.com/PacktPublishing...

One-hot encoding is a method used to represent categorical data, where each category is represented by a binary variable. The binary variable takes a value of 1 if the category is present, or 0 otherwise.

The following table shows the one-hot encoded representation of the Smoker variable with the categories of Smoker and Non-Smoker:

Figure 2.1 – One-hot encoded representation of the Smoker variable

As shown in Figure 2.1, from the Smoker variable, we can derive a binary variable for Smoker, which shows the value of 1 for smokers, or the binary variable for Non-Smoker, which takes the value of 1 for those who do not smoke.

For the Color categorical variable with the values of red, blue, and green, we can create three variables called red, blue, and green. These variables will be assigned a value of 1 if the observation corresponds to the respective color, and 0 if it does not.

A categorical...

One-hot encoding represents each variable’s category with a binary variable. Hence, one-hot encoding of highly cardinal variables or datasets with multiple categorical features can expand the feature space dramatically. This, in turn, may increase the computational cost of using machine learning models or deteriorate their performance. To reduce the number of binary variables, we can perform one-hot encoding of the most frequent categories. One-hot encoding the top categories is equivalent to treating the remaining, less frequent categories as a single, unique category.

In this recipe, we will implement one-hot encoding of the most popular categories using pandas, Scikit-learn, and feature-engine.

How to do it...

First, let’s import the necessary Python libraries and get the dataset ready:

1. Import the required Python libraries, functions, and classes:

   import pandas as pd
   import numpy as np
   from sklearn...

In count with counts or frequency of observations” or frequency encoding, we replace the categories with the count or the fraction of observations showing that category. That is, if 10 out of 100 observations show the blue category for the Color variable, we would replace blue with 10 when doing count encoding, or with 0.1 if performing frequency encoding. These encoding methods are useful when there is a relationship between the category frequency and the target. For example, in sales, the frequency of a product may indicate its popularity.

Note

If two different categories are present in the same number of observations, they will be replaced by the same value, which may lead to information loss.

In this recipe, we will perform count and frequency encoding using pandas and feature-engine.

How to do it...

We’ll start by encoding one variable with pandas and then we’ll automate the process...

Ordinal encoding consists of replacing the categories with digits from 1 to k (or 0 to k-1, depending on the implementation), where k is the number of distinct categories of the variable. The numbers are assigned arbitrarily. Ordinal encoding is better suited for non-linear machine learning models, which can navigate through arbitrarily assigned numbers to find patterns that relate to the target.

In this recipe, we will perform ordinal encoding using pandas, scikit-learn, and feature-engine.

How to do it...

First, let’s make the import and prepare the dataset:

1. Import pandas and the data split function:

   import pandas as pd
   from sklearn.model_selection import train_test_split

2. Let’s load the Credit Approval dataset and divide it into train and test sets:

   data = pd.read_csv("credit_approval_uci.csv")
   X_train, X_test, y_train, y_test = train_test_split(
       data.drop(labels=["target...

In the previous recipe, we replaced categories with integers, which were assigned arbitrarily. We can also assign integers to the categories given the target values. To do this, first, we calculate the mean value of the target per category. Next, we order the categories from the one with the lowest to the one with the highest target mean value. Finally, we assign digits to the ordered categories, starting with 0 to the first category up to k-1 to the last category, where k is the number of distinct categories.

This encoding method creates a monotonic relationship between the categorical variable and the response and therefore makes the variables more adequate for use in linear models.

In this recipe, we will encode categories while following the target value using pandas and feature-engine.

How to do it...

First, let’s import the necessary Python libraries and get the dataset ready:

1. Import the required...

Mean encoding or target encoding maps each category to the probability estimate of the target attribute. If the target is binary, the numerical mapping is the posterior probability of the target conditioned to the value of the category. If the target is continuous, the numerical representation is given by the expected value of the target given the value of the category.

In its simplest form, the numerical representation for each category is given by the mean value of the target variable for a particular category group. For example, if we have a City variable, with the categories of London, Manchester, and Bristol, and we want to predict the default rate (the target takes values of 0 and 1); if the default rate for London is 30%, we replace London with 0.3; if the default rate for Manchester is 20%, we replace Manchester with 0.2; and so on. If the target is continuous – say we want to predict income – then we would replace London,...

Weight of Evidence (WoE) was developed primarily for credit and financial industries to facilitate variable screening and exploratory analysis and to build more predictive linear models to evaluate the risk of loan defaults.

The WoE is computed from the basic odds ratio:

Here, positive and negative refer to the values of the target being 1 or 0, respectively. The proportion of positive cases per category is determined as the sum of positive cases per category group divided by the total positive cases in the training set. The proportion of negative cases per category is determined as the sum of negative cases per category group divided by the total number of negative observations in the training set.

WoE has the following characteristics:

• WoE = 0 if p(positive) / p(negative) = 1; that is, if the outcome is random
• WoE > 0 if p(positive) > p(negative)
• WoE < 0 if p(negative) > p(positive)

This allows us to...

Rare categories are those present only in a small fraction of the observations. There is no rule of thumb to determine how small a small fraction is, but typically, any value below 5% can be considered rare.

Infrequent labels often appear only on the train set or only on the test set, thus making the algorithms prone to overfitting or being unable to score an observation. In addition, when encoding categories to numbers, we only create mappings for those categories observed in the train set, so we won’t know how to encode new labels. To avoid these complications, we can group infrequent categories into a single category called Rare or Other.

In this recipe, we will group infrequent categories using pandas and feature-engine.

How to do it...

First, let’s import the necessary Python libraries and get the dataset ready:

1. Import the necessary Python libraries, functions, and classes:

   import numpy as np
   import pandas...

Binary encoding uses binary code – that is, a sequence of zeroes and ones – to represent the different categories of the variable. How does it work? First, the categories are arbitrarily replaced with ordinal numbers, as shown in the intermediate step of the following table. Then, those numbers are converted into binary code. For example, integer 1 can be represented with the sequence of 1-0, integer 2 with 0-1, integer 3 with 1-1, and integer 0 with 0-0. The digits in the two positions of the binary string become the columns, which are the encoded representations of the original variable:

Figure 2.10 – Table showing the steps required for binary encoding the color variable

Binary encoding encodes the data in fewer dimensions than one-hot encoding. In our example, the Color variable would be encoded into k-1 categories by one-hot encoding – that is, three variables – but with binary encoding...