Discovering wrapper, hybrid, and advanced feature selection methods
The feature selection methods studied so far are computationally inexpensive because they require no model fitting or fitting simpler white-box models. In this section, we will learn about other, more exhaustive methods with many possible tuning options. The categories of methods included here are as follows:
- Wrapper: Exhaustively look for the best subset of features by fitting an ML model using a search strategy that measures improvement on a metric.
- Hybrid: A method that combines embedded and filter methods with wrapper methods.
- Advanced: A method that doesn't fall into any of the previously discussed categories. Examples include dimensionality reduction, model-agnostic feature importance, and GAs.
And now, let's get started with wrapper methods!
Wrapper methods
The concept behind wrapper methods is reasonably simple: evaluate different subsets of features on the ML model and choose the one that achieves...
