Performing winsorization

Winsorization, or winsorizing, is the process of transforming the data by limiting the extreme values, that is, the outliers, to a certain arbitrary value, closer to the mean of the distribution. Winsorizing is different from trimming because the extreme values are not removed, but are instead replaced by other values. A typical strategy involves setting outliers to a specified percentile.

For example, with 90% winsorization, we set all data below the 5^th percentile to the value at the 5^th percentile and all data above the 95^th percentile to the value at the 95^th percentile. Winsorization is symmetric; therefore, the winsorized mean of a symmetric distribution provides an unbiased representation of the distribution of the variable. In this recipe, we will perform winsorization using pandas, NumPy, and Feature-engine.

...