Creating New Features

We can create valuable features by combining two or more variables. For example, in finance, the disposable income, which is the total income minus the acquired debt for any one month, might be more predictive of credit risk than just the income. Similarly, the total acquired debt of a person across financial products, such as a car loan, a mortgage, and credit cards, might be more predictive of credit risk than any debt considered individually. In these examples, we used domain knowledge of the data to craft the new variables, and the new variables were created by adding or subtracting existing features.

In some cases, a variable may not have a linear or monotonic relationship with the target, but a polynomial combination might. For example, if our variable has a quadratic relationship with the target, , we can convert that into a linear relationship by squaring the original variable. We can also obtain better variable relationships with the target by transforming...