Understanding model learning curves and trade-offs

In machine learning (ML) problems, we are always trying to find more data to improve our models, but as you can imagine, there comes a time when we reach a point of diminishing returns. It is very hard to know when you have reached that point, but you can get indications by looking at the learning curves. Fortunately, DataRobot makes that task easy by automatically building these learning curves. When DataRobot starts building models, it first tries a broad range of algorithms on small samples of data. Promising models are then built with bigger sample sizes, and so on.

In this process, we discover how much performance improvement happens as more data is added. To look at the learning curves, you can click on the Learning Curves menu item at the top of the screen, as seen in the following screenshot:

Figure 7.21 – Model learning curves

You can see the different model types on the right-hand side of...