Eliminating features recursively in a regression model

A popular wrapper method is RFE. This method starts with all features, removes the lowest weighted one (based on a coefficient or feature importance measure), and repeats the process until the best-fitting model has been identified. When a feature is removed, it is given a ranking reflecting the point at which it was removed.

RFE can be used for both regression models and classification models. We will start by using it in a regression model:

1. We import the necessary libraries, three of which we have not used yet: the RFE, RandomForestRegressor, and LinearRegression modules from scikit-learn:

   import pandas as pd
   from feature_engine.encoding import OneHotEncoder
   from sklearn.model_selection import train_test_split
   from sklearn.preprocessing import StandardScaler
   from sklearn.feature_selection import RFE
   from sklearn.ensemble import RandomForestRegressor
   from sklearn.linear_model import LinearRegression

2. Next, we load...