Multivariate Adaptive Regression Splines (MARS)
How would you like a modeling technique that provides all of the following?
- Offers the flexibility to build linear and nonlinear models for both regression and classification
- Can support variable interaction terms
- Is simple to understand and explain
- Requires little data preprocessing
- Handles all types of data: numeric, factors, and so on
- Performs well on unseen data, that is, it does well in bias-variance trade-off
If that all sounds appealing, then I cannot recommend the use of MARS models enough. The method was brought to my attention several months ago, and I have found it to perform extremely well. In fact, in a recent case of mine, it outperformed both a random forest and boosted trees on test/validation data. It has quickly become my baseline model and all others are competitors. The other benefit I've seen is that it has negated much of the feature engineering I was doing. Much of that was using Weight-of-Evidence (WOE) and Information Values...
