What is feature importance?
Feature importance refers to the extent to which each feature contributes to the final output of a model. For linear models, it’s easier to determine the importance since coefficients clearly indicate the contributions of each feature. However, this isn’t always the case for non-linear models.
To simplify the concept, let’s compare model classes to various team sports. In some sports, it’s easy to identify the players who have the greatest impact on the outcome, while in others, it isn’t. Let’s consider two sports as examples:
- Relay race: In this sport, each runner covers equal distances, and the race’s outcome largely depends on the speed at which they complete their part. Thus, it’s easy to separate and quantify each racer’s contributions. A relay race is similar to a linear model since the race’s outcome is a linear combination of independent components.
- Basketball...
