Mission accomplished
The mission was to provide an objective evaluation of the garbage classification model for the municipal recycling plant. The predictive performance on out-of-sample validation images was dismal! You could have stopped there, but then you would not have known how to make a better model.
However, the predictive performance evaluation was instrumental in deriving specific misclassifications, as well as correct classifications, to assess using other interpretation methods. To this end, you ran a comprehensive suite of interpretation methods, including activation, gradient, perturbation, and backpropagation-based methods. The consensus between all the methods was that the model was having the following issues:
- Differentiating between the background and the objects
- Understanding that different objects share similar color hues
- Confounding lighting conditions, such as specular highlights as specific material characteristics, like with the wine...
