Performing Feature Scaling

Many machine learning algorithms are sensitive to the scale of the features. In particular, the coefficients of linear models depend on the scale of the feature; that is, changing the feature scale will change the coefficient’s value. In linear models, as well as and algorithms that depend on distance calculations, such as clustering and principal component analysis, features with bigger value ranges tend to dominate over features with smaller ranges. Therefore, having features within a similar scale allows us to compare feature importance and also helps algorithms converge faster, thus improving performance and training times.

Scaling techniques will divide the variables by some constant; therefore, it is important to highlight that no matter the scaling method, the shape of the variable distribution does not change. If what you want is to change the distribution shape, check out Chapter 3, Transforming Numerical Variables.

In this chapter,...