Visualization for performance assessment
Visualization is an important tool that helps us not only understand the characteristics of our data for modeling but also better assess the performance of our models. Visualization could provide complementary information to the aforementioned model performance metrics.
Summary metrics are not enough
There are summary statistics such as ROC-AUC and PR-AUC that provide a one-number summary of their corresponding curves for assessing the performance of classification models. Although these summaries are more reliable than many other metrics such as accuracy, they do not completely capture the characteristics of their corresponding curves. For example, two different models with different ROC curves can have the same or very close ROC-AUCs (Figure 4.6):
Figure 4.6 – Comparison of two arbitrary models with the same ROC-AUCs and different ROC curves
Comparing ROC-AUCs alone could result in deciding the equivalence...
