As we mentioned previously, none of these described methods will tell us precisely how to choose the input features by themselves. It is true that in some particular cases, if the correlations are strong enough, we could discard one or more features and just keep the ones that represent them by correlation. In general, feature engineering is a long and time-consuming task that became almost a separate field of study within machine learning.
There are automatic techniques to perform feature engineering, which are part of what is generically called Automatic Machine Learning (AutoML). The method consists of letting the computer try different feature sets, including combinations of them, and test the results until the best set is found. In spite of this, there is no general recipe for selecting features, and each problem has to be analyzed—in...
