Exploring Benchmarking Attribution Methods (BAM)

With various feature attribution XAI methods available today, it can be challenging to quantify which inputs are indeed important to a model due to a lack of ground truth data. Relying solely on visual assessment can be misleading. XAI methods that compute gradients insensitive to the input data fail to produce relevant explanations for desired target outputs, which can be detrimental in ML tasks, such as anomaly detection. For instance, research has shown that some XAI saliency methods are unable to demonstrate true feature importance and produce identical visual explanations or saliency maps despite randomizing a trained model’s parameters. For AI-assisted medical diagnosis, this can be risky and concerning when there is no change in explanations to a model’s prediction after randomizing a trained model’s data and parameters.

There is no value in providing false positive explanations to the target audience....