Blending models using a meta-model
The Netflix competition (which we discussed at length in Chapter 1) didn’t just demonstrate that averaging would be advantageous for difficult problems in a data science competition; it also brought about the idea that you can use a model to average your models’ results more effectively. The winning team, BigChaos, in their paper (Töscher, A., Jahrer, M., and Bell, R.M. The BigChaos Solution to the Netflix Grand Prize. Netflix prize documentation – 2009) made many mentions of blending, and provided many hints about its effectiveness and the way it works.
In a few words, blending is kind of a weighted averaging procedure where the weights used to combine the predictions are estimated by way of a holdout set and a meta-model trained on it. A meta-model is simply a machine learning algorithm that learns from the output of other machine learning models. Usually, a meta-learner is a linear model (but sometimes it can also...
