Summary

Some of the evaluation metrics for classification models require a binary classification model. If you are working with more than two classes, you will need to use one-versus-all. The one-versus-all approach builds one model for each class and tries to predict the probability that the input belongs to a specific class. You will then predict that the input belongs to the class where the model has the highest prediction probability.

ROC and ROC AUC only work with binary classification.

If you were wondering why we split our evaluation dataset into two, it's because X_test and y_test are used once for a final evaluation of the model's performance. You make use of them before putting your model into production to see how the model would perform in a production environment.

You have learned how to assess the quality of a regression model by observing how the loss changes. You saw examples using the MAE, and also learned of the existence of MSE.

...