Difficulties in ensemble learning

Although ensemble learning can greatly increase the performance of machine learning models, it comes at a cost. There are difficulties and drawbacks in correctly implementing it. Some of these difficulties and drawbacks will now be discussed.

Weak or noisy data

The most important ingredient of a successful model is the dataset. If the data contains noise or incomplete information, there is not a single machine learning technique that will generate a highly performant model.

Let's illustrate this with a simple example. Suppose we study populations (in the statistical sense) of cars and we gather data about the color, shape, and manufacturer. It is difficult to generate a very accurate...