A performant predictive model is one that produces reliable and satisfying predictions on new data. There are two situations where the model will fail to consistently produce good predictions, and both depend on how the model is trained. A poorly trained model will result in underfitting, while an overly trained model will result in overfitting.
Identifying and correcting poor performances
Underfitting
Underfitting means that the model was poorly trained. Either the training dataset did not have enough information to infer strong predictions, or the algorithm that trained the model on the training dataset was not adequate for the context. The algorithm was not well parameterized or simply inadequate for the data.
If we measure the prediction error...
