Dimension reduction – feature transformations versus feature selection versus feature construction

In the last section, I mentioned how we could squish datasets to have fewer columns to describe data in new ways. This sounds similar to the concept of feature selection: removing columns from our original dataset to create a different, potentially better, views of our dataset by cutting out the noise and enhancing signal columns. While both feature selection and feature transformation are methods of performing dimension reduction, it is worth mentioning that they could not be more different in their methodologies. 

Feature selection processes are limited to only being able to select features from the original set of columns, while feature transformation algorithms use these original columns and combine them in useful ways to create new columns that are better at describing the data than any single column from the original dataset. Therefore, feature selection methods reduce dimensions by isolating...