Attribution-Based Explainability in Medical Imaging: A Critical Review on Explainable Computer Vision (X-CV) Techniques and Their Applications in Medical AI
Abstract
1. Introduction
- A systematic taxonomy of X-CV methods, categorizing them into gradient-based, perturbation-based, CAM-based, backpropagation-based, and meta-attribution approaches, with detailed technical explanations.
- A thorough review of their applications across radiology, dermatology, pathology and ophthalmology, supported by real-world examples and performance metrics.
- An in-depth evaluation of validation methods, combining human assessment, axiomatic properties and quantitative metrics to assess clinical utility.
- Identification of key challenges, such as computational complexity and data variability, with proposed future directions for improving fairness, standardisation and multimodal integration.
2. Fundamentals of Attribution in Explainability
2.1. Attribution-Based X-CV Methods
Algorithm 1 Unified Attribution-Based X-CV Pipeline |
Require: Dataset Ensure:
Attribution maps 1:
Preprocess dataset: resize images to 2: Train or load model f (e.g., ResNet50) 3: for to n do 4: , predict class 5: Choose attribution method type: 6: Gradient-based / Perturbation / CAM / Backprop / Meta 7: Apply selected attribution method to x 8: Store result 9: end for 10: Visualize or evaluate |
2.1.1. Gradient-Based Methods
- Saliency Map: Saliency maps compute [29] the absolute gradient of the model’s output score (e.g., the likelihood of a disease class) with respect to input pixels in order to illustrate sensitivity to pixel changes. An region impacted by pneumonia is indicated by high gradient values on a chest X-ray, and a heatmap is produced by backpropagating from the output to the input layer; areas that are brighter indicate [12] greater effect. Their benefits include low computing cost and model-agnostic applicability, but they suffer [9] from saturation and noise issues that may point out irrelevant regions if gradients vanish. In medical imaging, saliency maps facilitate rapid viewing; nevertheless, because of their complexity, they require thorough validation.
- Vanilla Gradients: Similar to saliency maps [29], vanilla gradients [30] employ the output’s raw gradients with respect to the input without any modifications. The way they maintain gradient signals, which indicate whether pixel changes lead to a better or lower output score, is where they differ. On an MRI, positive gradients could highlight a tumour’s core. They are straightforward but vulnerable [12,31] to noise and lack resilience in deep networks. Their application in medical imaging, where they are commonly employed as a foundation for more complex methods, is limited by these issues.
- Integrated Gradients: The integrated gradients function computes gradients through input-baseline differentiation while addressing [20] gradient saturation by creating gradients that span from baseline images to input images. The technique provides precise micro-calcification detection in mammography images. The method shows three main characteristics including reduced noise levels together with complete sensitivity to all features and axiomatic fairness [11,20] which guarantees that output changes match their corresponding attributions. The main disadvantage of this approach is its high computational cost which demands multiple gradient evaluations. Medical imaging applications benefit from its precision to detect lesions and define their borders.
- Guided Backpropagation: The approach reduces [32] noise levels while improving visualisation by spreading only positive gradients through positive activations which modifies backpropagation. A retinal scan could show microaneurysms in diabetic retinopathy. The modified ReLU activations during backpropagation produce heatmaps that show better readability than standard gradients. The system produces enhanced visual displays [9] with strong emphasis on positive elements yet it has the potential to distort certain information and hide negative data points. Medical imaging benefits from this method to achieve better visualisation yet requires proper evaluation to prevent biased results.
- SmoothGrad: The SmoothGrad algorithm reduces gradient-based map noise through multiple rounds of input image noise application (e.g., Gaussian noise addition). The CT scan would generate a continuous heat map of the lung nodule through its application. The method works by sampling noisy inputs to calculate their gradients before averaging the results [33]. The method provides better visual clarity and robustness [31] although it increases computational cost and reduces fine details. The method improves the clinical review consistency of gradient-based medical imaging explanations.
Algorithm 2 Gradient-Based Attribution Methods |
Require:
Trained model f, input image Ensure: Saliency maps 1: Preprocess image x (resize to ) 2: Compute model prediction 3: for each gradient-based method do 4: if method == Vanilla Gradients then 5: 6: else if method == Integrated Gradients then 7: 8: else if method == SmoothGrad then 9: Add Gaussian noise to x, average gradients over N noisy samples 10: else if method == Saliency Map then 11: 12: else if method == Guided Backpropagation then 13: Modify ReLU in backprop and compute gradients 14: end if 15: end for |
2.1.2. Perturbation-Based Methods
- Occlusion Sensitivity: This technique measures [34] the impact of obstructing each region on the output score by sliding a patch, such as a gray square, across the image. A decrease in the likelihood of pneumonia in an X-ray when a lung region is obscured shows how important it is. It creates a rough heatmap by methodically altering areas and documenting output variations. Although it is sensitive to patch size and computationally costly, its properties include [11] intuitiveness and independence from model internals. It aids in locating important areas in medical imaging but has trouble with fine-grained details.
- LIME for vision: By altering superpixels (image segments) and fitting a basic model (such as linear regression) to forecast [35] the output of the original model, LIME approximates a model’s behavior locally. It could draw attention to the uneven border of a melanoma in a skin lesion image. The surrogate model is trained by creating perturbed images and weighing them according to how similar they are to the original. Flexibility and local fidelity are among its advantages; however, superpixel segmentation is computationally demanding and may introduce errors [36]. Although LIME is less accurate than gradient-based techniques, its interpretability helps non-experts in medical imaging.
- SHAP for vision: By calculating the marginal contribution of each segment across all possible combinations, SHAP uses Shapley values [37] from game theory to assign importance to superpixels. A retinal scan may reveal hemorrhages that are the basis for a diagnosis. It generates additive attributions by using sampling to approximate Shapley values. Although its high computational cost frequently necessitates approximations [11], which reduce accuracy, its properties include theoretical fairness, consistency, and robustness to complex interactions. Although difficult to scale, SHAP’s thorough explanations are helpful in complex medical imaging cases.
Algorithm 3 Perturbation-Based Attribution Methods |
Require:
Trained model f, input image x Ensure:
Attribution map 1: Preprocess image x 2: for each perturbation method do 3: if method == Occlusion Sensitivity then 4: for each patch in x do 5: Occlude patch 6: 7: Update with 8: end for 9: else if method == LIME then 10: Generate N perturbed samples around x 11: Train local linear model g to approximate f locally 12: weights from g 13: else if method == SHAP then 14: Compute Shapley values for pixel coalitions 15: average marginal contribution across subsets 16: end if 17: end for |
2.1.3. Meta Attribution-Based Methods
- RISE (Randomized Input Sampling): RISE produces importance maps through a process where it applies random masks to images and calculates [38] weighted average output scores based on mask presence. The approach demonstrates model independence and noise resistance yet demands significant computational resources and generates maps at a general level. The flexible nature of RISE makes it suitable [36] for various medical imaging tasks yet it fails to detect small features with precision.
- External Perturbations: This method identifies [39] the minimal image region needed to preserve a model’s prediction by optimizing a mask to maximize the output score. In a CT scan, it might isolate a lung nodule’s core. It works by iteratively adjusting the mask using gradient descent, balancing attribution sparsity and prediction fidelity. This method differs from other ‘Perturbation’ methods like occlusion sensitivity and LIME by optimizing that mask through gradient descent to identify minimal image regions critical for predictions, combining perturbation with gradient-based techniques. Properties include high specificity and focus on critical regions, but optimisation can be unstable [11], and it is computationally demanding. In medical imaging, it excels at pinpointing key features for surgical planning.
Algorithm 4 Meta-Attribution Methods |
Require: Model f, image x Ensure: Attribution map 1: for method in Meta-Attribution Methods do 2: if method == RISE then 3: Generate N random binary masks 4: for each do 5: 6: 7: end for 8: 9: else if method == External Perturbation then 10: Modify semantic/feature-level inputs 11: Measure change in , update 12: end if 13: end for |
2.1.4. Class Activation Map (CAM)-Based Methods
- GradCAM: In order to create a heatmap, Grad-CAM calculates gradients of the target class score [40] in relation to the feature maps of the final convolutional layer, averages them to determine neuron weights, and then combines them with feature maps. It could draw attention to a brain tumour in an MRI. Robustness, compatibility with any CNN, and coarse but dependable localisation [9] are among its attributes. Although fine details are limited by its lower resolution, its balance of clarity and generality makes it a popular choice for medical imaging.
- GradCAM++: It is an extension of Grad-CAM that improves [41] localisation for multiple instances of a class by using higher-order gradients to weight pixels within feature maps. GradCAM++ mapping improves heatmap accuracy by adding pixel-level gradient contributions. Although it is a little more complicated, its properties include enhanced granularity and resilience to occlusions [9]. It is useful for complex scenes, such as multifocal diseases, in medical imaging.
- ScoreCAM: By using each feature map as a mask to calculate its contribution to the output, Score-CAM substitutes [42] activation scores for gradients. It could draw attention to classification in a mammogram without gradient noise. Feature maps are normalized, then used as masks, and scores are aggregated. High visual quality and gradient-free robustness are among its attributes, but it requires [7] a lot of computing power. It is recommended for noise-sensitive tasks in medical imaging.
- XGradCAM: Grad-CAM [40] is altered by XGrad-CAM to meet axiomatic requirements [43] such as conservation and normalizing weights to guarantee that attributions match the output score. It guarantees balanced hemorrhage highlighting for a retinal scan. In order to satisfy theoretical constraints, gradient weighting is modified. Fairness and stability are among its attributes, and its resolution is comparable [16] to that of Grad-CAM. It increases the credibility of explanations in medical imaging.
Algorithm 5 Class Activation Mapping-Based Methods |
Require: CNN model f, input image x Ensure:
Class activation map 1: Preprocess image x 2: Obtain feature maps F from last convolutional layer 3: for each CAM method do 4: if method == Grad-CAM then 5: Compute gradients 6: Compute weights 7: 8: else if method == Grad-CAM++ then 9: Use higher-order gradients for refined 10: else if method == Score-CAM then 11: Mask input using , forward pass for each 12: Weight each by softmax scores 13: else if method == XGrad-CAM then 14: Normalize and use absolute gradients 15: end if 16: end for |
2.1.5. Backpropagation-Based Methods
- LRP: LRP distributes output prediction scores throughout neural network layers by following conservation rules which maintain overall relevance at each stage [44]. The method distributes each neuron’s importance through proportional values to the neurons located above it without requiring [8] gradient calculations. In medical imaging perspective, LRP is robust because it generates detailed spatial attributions that support disease localisation although rule adjustment might be needed for various design configurations.
- DeepLIFT: The output variance in DeepLIFT is tracked [45] through “contribution scores” that measure how input modifications affect each neuron’s activity relative to its reference activation. The method maintains consistent attribution by propagating these discrepancies from input to output while considering both positive and negative contributions. In this method, selection of baseline images requires attention because DeepLIFT facilitates contrastive interpretation in medical imaging which helps distinguish normal from diseased states.
Algorithm 6 Backpropagation-Based Attribution Methods |
Require: Model f, input image x Ensure:
Relevance map 1: Forward pass: 2: Initialize relevance: 3: for each layer l from output to input do 4: if method == LRP then 5: Redistribute R using relevance conservation rules 6 else if method == DeepLIFT then 7: Compute reference activation 8: 9: end if 10: end for |
2.2. Non-Attribution-Based X-CV Methods
3. Applications in Medical Imaging
3.1. Specific Medical Imaging Modalities
3.1.1. Radiology (X-Rays, MRI, CT)
3.1.2. Dermatology (Dermoscopic Images)
3.1.3. Pathology (Histopathology Images)
3.1.4. Ophthalmology (OCT, Fundus Images)
3.1.5. Comparative Analysis of Other Performance Metrics
3.2. Clinical Tasks and Applications
3.2.1. Disease Diagnosis and Classification
3.2.2. Treatment Response Prediction
3.2.3. Lesion Detection and Segmentation
3.2.4. Biomarker Discovery
3.2.5. Clinical Workflow Examples
3.3. Validation and Evaluation of Attribution Methods
3.3.1. Human Evaluation
3.3.2. Axiomatic Properties
3.3.3. Quantitative Metrics
3.3.4. Challenges in Validation
3.4. Future Directions
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Lehman, C.D.; Wellman, R.D.; Buist, D.S.M.; Kerlikowske, K.; Tosteson, A.N.A.; Miglioretti, D.L. Diagnostic Accuracy of Digital Screening Mammography With and Without Computer-Aided Detection. JAMA Intern. Med. 2015, 175, 1828. [Google Scholar] [CrossRef] [PubMed]
- Rajpurkar, P.; Irvin, J.; Zhu, K.; Yang, B.; Mehta, H.; Duan, T.; Ding, D.; Bagul, A.; Langlotz, C.; Shpanskaya, K.; et al. CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning. arXiv 2017, arXiv:1711.05225. [Google Scholar]
- Hosny, A.; Parmar, C.; Quackenbush, J.; Schwartz, L.H.; Aerts, H.J.W.L. Artificial Intelligence in Radiology. Nat. Rev. Cancer 2018, 18, 500–510. [Google Scholar] [CrossRef]
- Topol, E.J. High-Performance Medicine: The Convergence of Human and Artificial Intelligence. Nat. Med. 2019, 25, 44–56. [Google Scholar] [CrossRef]
- Yi, X.; Walia, E.; Babyn, P. Generative Adversarial Network in Medical Imaging: A Review. Med. Image Anal. 2019, 58, 101552. [Google Scholar] [CrossRef]
- Wegmayr, V.; Aitharaju, S.; Buhmann, J.M. Classification of Brain MRI with Big Data and Deep 3D Convolutional Neural Networks. Proc. SPIE 2018, 10575, 105751S. [Google Scholar] [CrossRef]
- Huang, S.-C.; Pareek, A.; Seyyedi, S.; Banerjee, I.; Lungren, M.P. Fusion of Medical Imaging and Electronic Health Records Using Deep Learning: A Systematic Review and Implementation Guidelines. NPJ Digit. Med. 2020, 3, 136. [Google Scholar] [CrossRef]
- Obermeyer, Z.; Powers, B.; Vogeli, C.; Mullainathan, S. Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations. Science 2019, 366, 447–453. [Google Scholar] [CrossRef] [PubMed]
- Holzinger, A.; Langs, G.; Denk, H.; Zatloukal, K.; Müller, H. Causability and Explainability of Artificial Intelligence in Medicine. WIREs Data Min. Knowl. Discov. 2019, 9, e1312. [Google Scholar] [CrossRef] [PubMed]
- Tonekaboni, S.; Joshi, S.; McCradden, M.D.; Goldenberg, A. What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use. arXiv 2019, arXiv:1905.05134. [Google Scholar] [CrossRef]
- Amann, J.; Blasimme, A.; Vayena, E.; Frey, D.; Madai, V.I. Explainability for Artificial Intelligence in Healthcare: A Multidisciplinary Perspective. BMC Med. Inf. Decis. Mak. 2020, 20, 310. [Google Scholar] [CrossRef]
- Reyes, M.; Meier, R.; Pereira, S.; Silva, C.A.; Dahlweid, F.-M.; von Tengg-Kobligk, H.; Summers, R.M.; Wiest, R. On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. Radiol. Artif. Intell. 2020, 2, e190043. [Google Scholar] [CrossRef]
- Vayena, E.; Blasimme, A.; Cohen, I.G. Machine Learning in Medicine: Addressing Ethical Challenges. PLoS Med. 2018, 15, e1002689. [Google Scholar] [CrossRef]
- Goodman, B.; Flaxman, S. European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”. AI Mag. 2017, 38, 50–57. [Google Scholar] [CrossRef]
- Diprose, W.K.; Buist, N.; Hua, N.; Thurier, Q.; Shand, G.; Robinson, R. Physician Understanding, Explainability, and Trust in a Hypothetical Machine Learning Risk Calculator. J. Am. Med. Inf. Assoc. 2020, 27, 592–600. [Google Scholar] [CrossRef] [PubMed]
- Explainable Computer Vision: Where Are We and Where Are We Going? In Proceedings of the European Conference on Computer Vision Workshops, Milan, Italy, 29 September–4 October 2024.
- Liu, C.-F.; Chen, Z.-C.; Kuo, S.-C.; Lin, T.-C. Does AI Explainability Affect Physicians’ Intention to Use AI? Int. J. Med. Inf. 2022, 168, 104884. [Google Scholar] [CrossRef] [PubMed]
- Korteling, J.E.; van de Boer-Visschedijk, G.C.; Blankendaal, R.A.; Boonekamp, R.C.; Eikelboom, A.R. Human- versus Artificial Intelligence. Front. Artif. Intell. 2021, 4, 622364. [Google Scholar] [CrossRef]
- Yang, G.; Ye, Q.; Xia, J. Unbox the Black-Box for the Medical Explainable AI via Multi-Modal and Multi-Centre Data Fusion: A Mini-Review, Two Showcases and Beyond. Inf. Fusion 2022, 77, 29–52. [Google Scholar] [CrossRef]
- Sundararajan, M.; Taly, A.; Yan, Q. Axiomatic Attribution for Deep Networks. arXiv 2017, arXiv:1703.01365. [Google Scholar] [CrossRef]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. arXiv 2016, arXiv:1610.02391. [Google Scholar]
- Suara, S.; Jha, A.; Sinha, P.; Sekh, A.A. Is Grad-CAM Explainable in Medical Images? arXiv 2023, arXiv:2307.10506. [Google Scholar] [CrossRef]
- Lee, K.-S.; Kim, J.Y.; Jeon, E.-t.; Choi, W.S.; Kim, N.H.; Lee, K.Y. Evaluation of Scalability and Degree of Fine-Tuning of Deep Convolutional Neural Networks for COVID-19 Screening on Chest X-ray Images Using Explainable Deep-Learning Algorithm. J. Med. Imaging 2020, 7, 067501. [Google Scholar] [CrossRef] [PubMed]
- Bhati, D.; Neha, F.; Amiruzzaman, M. A Survey on Explainable Artificial Intelligence (XAI) Techniques for Visualizing Deep Learning Models in Medical Imaging. J. Imaging 2024, 10, 239. [Google Scholar] [CrossRef]
- Linardatos, P.; Papastefanopoulos, V.; Kotsiantis, S. Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy 2021, 23, 18. [Google Scholar] [CrossRef]
- van der Velden, B.H.M.; Kuijf, H.J.; Gilhuijs, K.G.A.; Viergever, M.A. Explainable Artificial Intelligence (XAI) in Deep Learning-Based Medical Image Analysis. Med. Image Anal. 2022, 79, 102470. [Google Scholar] [CrossRef]
- Lundberg, S.M.; Erion, G.; Chen, H.; DeGrave, A.; Prutkin, J.M.; Nair, B.; Katz, R.; Himmelfarb, J.; Bansal, N.; Lee, S.-I. From Local Explanations to Global Understanding with Explainable AI for Trees. Nat. Mach. Intell. 2020, 2, 56–67. [Google Scholar] [CrossRef] [PubMed]
- Arrieta, A.B.; Díaz-Rodríguez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; Garcia, S.; Gil-Lopez, S.; Molina, D.; Benjamins, R.; et al. Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges Toward Responsible AI. Inf. Fusion 2020, 58, 82–115. [Google Scholar] [CrossRef]
- Simonyan, K.; Vedaldi, A.; Zisserman, A. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv 2014, arXiv:1312.6034. [Google Scholar] [CrossRef]
- Yuan, R.; Gower, R.M.; Lazaric, A. A General Sample Complexity Analysis of Vanilla Policy Gradient. arXiv 2022, arXiv:2107.11433. [Google Scholar] [CrossRef]
- Ancona, M.; Ceolini, E.; Öztireli, C.; Gross, M. Gradient-Based Attribution Methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning; Samek, W., Montavon, G., Vedaldi, A., Hansen, L., Müller, K.-R., Eds.; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2019; Volume 11700, pp. 169–191. [Google Scholar] [CrossRef]
- Springenberg, J.T.; Dosovitskiy, A.; Brox, T.; Riedmiller, M. Striving for Simplicity: The All Convolutional Net. arXiv 2015, arXiv:1412.6806. [Google Scholar] [CrossRef]
- Smilkov, D.; Thorat, N.; Kim, B.; Viégas, F.; Wattenberg, M. SmoothGrad: Removing Noise by Adding Noise. arXiv 2017, arXiv:1706.03825. [Google Scholar] [CrossRef]
- Zeiler, M.D.; Fergus, R. Visualizing and Understanding Convolutional Networks. Proc. Eur. Conf. Comput. Vis. 2014, 818–833. [Google Scholar] [CrossRef]
- Ribeiro, M.T.; Singh, S.; Guestrin, C. “Why Should I Trust You?” Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 1135–1144. [Google Scholar] [CrossRef]
- Lipton, Z.C. The Mythos of Model Interpretability. Commun. ACM 2018, 61, 36–43. [Google Scholar] [CrossRef]
- Lundberg, S.M.; Lee, S.-I. A Unified Approach to Interpreting Model Predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS 2017), Long Beach, CA, USA, 4–9 December 2017; pp. 4765–4774. Available online: https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf (accessed on 5 May 2025).
- Petsiuk, V.; Das, A.; Saenko, K. RISE: Randomized Input Sampling for Explanation of Black-Box Models. arXiv 2018, arXiv:1806.07421. [Google Scholar] [CrossRef]
- Chang, C.-H.; Creager, E.; Goldenberg, A.; Duvenaud, D. Explaining Image Classifiers by Counterfactual Generation. arXiv 2019, arXiv:1807.08024. [Google Scholar] [CrossRef]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 618–626. [Google Scholar] [CrossRef]
- Chattopadhay, A.; Sarkar, A.; Howlader, P.; Balasubramanian, V.N. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA, 12–15 March 2018; pp. 839–847. [Google Scholar] [CrossRef]
- Wang, H.; Wang, Z.; Du, M.; Yang, F.; Zhang, Z.; Ding, S.; Mardziel, P.; Hu, X. Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 14–19 June 2020; pp. 24–25. [Google Scholar] [CrossRef]
- Fu, R.; Hu, Q.; Dong, X.; Guo, Y.; Gao, Y.; Li, B. Axiom-Based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs. arXiv 2020, arXiv:2008.02312. [Google Scholar]
- Binder, A.; Bach, S.; Montavon, G.; Klauschen, F.; Müller, K.-R.; Samek, W. Layer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers. arXiv 2016, arXiv:1604.00825. [Google Scholar] [CrossRef]
- Shrikumar, A.; Greenside, P.; Shcherbina, A.; Kundaje, A. Learning Important Features Through Propagating Activation Differences. arXiv 2017, arXiv:1704.02685. [Google Scholar] [CrossRef]
- Kim, B.; Wattenberg, M.; Gilmer, J.; Cai, C.J.; Wexler, J.; Viégas, F.; Sayres, R. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 10–15 July 2018; pp. 2668–2677. Available online: https://proceedings.mlr.press/v80/kim18a.html (accessed on 5 May 2025).
- Olah, C.; Mordvintsev, A.; Schubert, L. The Building Blocks of Interpretability. Distill 2018. Available online: https://distill.pub/2018/building-blocks/ (accessed on 5 May 2025).
- Koh, P.W.; Liang, P. Understanding Black-Box Predictions via Influence Functions. In Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia, 6–11 August 2017; pp. 1885–1894. Available online: https://proceedings.mlr.press/v70/koh17a.html (accessed on 5 May 2025).
- Pruthi, G.; Liu, F.; Sundararajan, M.; Kale, S. Estimating Training Data Influence by Tracing Gradient Descent. In Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 6–12 December 2020; pp. 19920–19930. Available online: https://papers.nips.cc/paper/2020/file/e6385d39ec9394f2f3a354d9d2b88eec-Paper.pdf (accessed on 5 May 2025).
- Rudin, C. Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead. Nat. Mach. Intell. 2019, 1, 206–215. [Google Scholar] [CrossRef]
- Doshi-Velez, F.; Kim, B. Towards a Rigorous Science of Interpretable Machine Learning. arXiv 2017, arXiv:1702.08608. [Google Scholar] [CrossRef]
- Yosinski, J.; Clune, J.; Nguyen, A.; Fuchs, T.; Lipson, H. Understanding Neural Networks Through Deep Visualization. arXiv 2015, arXiv:1506.06579. [Google Scholar] [CrossRef]
- Panwar, H.; Gupta, P.K.; Siddiqui, M.K.; Morales-Menendez, R.; Bhardwaj, P.; Singh, V. A Deep Learning and Grad-CAM Based Color Visualization Approach for Fast Detection of COVID-19 Cases Using Chest X-ray and CT-Scan Images. Chaos Solitons Fractals 2020, 140, 110190. [Google Scholar] [CrossRef]
- Sun, J.; Shi, W.; Giuste, F.O.; Vaghani, Y.S.; Tang, L.; Wang, M.D. Improving Explainable AI with Patch Perturbation-Based Evaluation Pipeline: A COVID-19 X-ray Image Analysis Case Study. Sci. Rep. 2023, 13, 19488. [Google Scholar] [CrossRef]
- Ganie, S.M.; Dutta Pramanik, P.K. Interpretable Lung Cancer Risk Prediction Using Ensemble Learning and XAI Based on Lifestyle and Demographic Data. Comput. Biol. Chem. 2025, 117, 108438. [Google Scholar] [CrossRef]
- Eitel, F.; Soehler, E.; Bellmann-Strobl, J.; Brandt, A.U.; Ruprecht, K.; Giess, R.M.; Kuchling, J.; Asseyer, S.; Weygandt, M.; Haynes, J.-D.; et al. Uncovering Convolutional Neural Network Decisions for Diagnosing Multiple Sclerosis on Conventional MRI Using Layer-Wise Relevance Propagation. Neuroimage Clin. 2019, 24, 102003. [Google Scholar] [CrossRef] [PubMed]
- Mohamed, M.M.; Mahesh, T.R.; Vinoth, K.V.; Suresh, G. Enhancing Brain Tumor Detection in MRI Images through Explainable AI Using Grad-CAM with Resnet 50. BMC Med. Imaging 2024, 24, 107. [Google Scholar] [CrossRef]
- Mohamed, E.; Sirlantzis, K.; Howells, G. A Review of Visualisation-as-Explanation Techniques for Convolutional Neural Networks and Their Evaluation. Displays 2022, 73, 102239. [Google Scholar] [CrossRef]
- Rahman, M.S.; Chowdhury, M.E.H.; Rahman, H.R.; Ahmed, M.U.; Kabir, M.A.; Roy, S.S.; Sarmun, R. Self-DenseMobileNet: A Robust Framework for Lung Nodule Classification Using Self-ONN and Stacking-Based Meta-Classifier. arXiv 2024, arXiv:2410.12584. [Google Scholar]
- Ihongbe, I.E.; Fouad, S.; Mahmoud, T.F.; Rajasekaran, A.; Bhatia, B. Evaluating Explainable Artificial Intelligence (XAI) Techniques in Chest Radiology Imaging Through a Human-Centered Lens. PLoS ONE 2024, 19, e0308758. [Google Scholar] [CrossRef] [PubMed]
- Matas, I.; Serrano, C.; Silva, F.; Serrano, A.; Toledo-Pastrana, T.; Acha, B. Clinically Inspired Enhance Explainability and Interpretability of an AI-Tool for BCC Diagnosis Based on Expert Annotation. arXiv 2024, arXiv:2407.00104. [Google Scholar]
- Shafiq, M.; Aggarwal, K.; Jayachandran, J.; Srinivasan, G.; Boddu, R.; Alemayehu, A. A Novel Skin Lesion Prediction and Classification Technique: ViT-GradCAM. Skin Res. Technol. 2024, 30, e70040. [Google Scholar] [CrossRef] [PubMed]
- Hamim, S.A.; Tamim, M.U.I.; Mridha, M.F.; Safran, M.; Che, D. SmartSkin-XAI: An Interpretable Deep Learning Approach for Enhanced Skin Cancer Diagnosis in Smart Healthcare. Diagnostics 2025, 15, 64. [Google Scholar] [CrossRef]
- Chanda, T.; Hauser, K.; Hobelsberger, S.; Bucher, T.-C.; Garcia, C.N.; Wies, C.; Kittler, H.; Tschandl, P.; Navarrete-Dechent, C.; Podlipnik, S.; et al. Dermatologist-like Explainable AI Enhances Trust and Confidence in Diagnosing Melanoma. Nat. Commun. 2024, 15, 524. [Google Scholar] [CrossRef] [PubMed]
- Hauser, K.; Kurz, A.; Haggenmüller, S.; Maron, R.C.; von Kalle, C.; Utikal, J.S.; Meier, F.; Hobelsberger, S.; Gellrich, F.F.; Sergon, M.; et al. Explainable Artificial Intelligence in Skin Cancer Recognition: A Systematic Review. Eur. J. Cancer 2022, 167, 54–69. [Google Scholar] [CrossRef]
- Shah, S.A.H.; Shah, S.T.H.; Khaled, R.; Buccoliero, A.; Shah, S.B.H.; Di Terlizzi, A.; Di Benedetto, G.; Deriu, M.A. Explainable AI-Based Skin Cancer Detection Using CNN, Particle Swarm Optimization and Machine Learning. J. Imaging 2024, 10, 332. [Google Scholar] [CrossRef]
- Liu, S. DALAResNet50 for Automatic Histopathology Breast Cancer Image Classification with DT Grad-CAM Explainability. arXiv 2024, arXiv:2308.13150. [Google Scholar]
- Kaur, A.; Kaushal, C.; Sandhu, J.K.; Damaševičius, R.; Thakur, N. Histopathological Image Diagnosis for Breast Cancer Diagnosis Based on Deep Mutual Learning. Diagnostics 2024, 14, 95. [Google Scholar] [CrossRef]
- Menon, A.; Singh, P.; Vinod, P.K.; Jawahar, C.V. Exploring Histological Similarities Across Cancers From a Deep Learning Perspective. Front. Oncol. 2022, 12, 842759. [Google Scholar] [CrossRef]
- Alabi, R.O.; Elmusrati, M.; Leivo, I.; Almangush, A.; Mäkitie, A.A. Machine Learning Explainability in Nasopharyngeal Cancer Survival Using LIME and SHAP. Sci. Rep. 2023, 13, 8984. [Google Scholar] [CrossRef]
- Patil, A.; Tamboli, D.; Meena, S.; Anand, D.; Sethi, A. Breast Cancer Histopathology Image Classification and Localization Using Multiple Instance Learning. In Proceedings of the IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE), Piscataway, NJ, USA, 15–16 November 2019; pp. 1–4. [Google Scholar] [CrossRef]
- Vasu, B.; Long, C. Iterative and Adaptive Sampling with Spatial Attention for Black-Box Model Explanations. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Snowmass, CO, USA, 1–5 March 2020; pp. 2960–2969. [Google Scholar]
- Salahuddin, Z.; Woodruff, H.C.; Chatterjee, A.; Lambin, P. Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods. Comput. Biol. Med. 2022, 140, 105111. [Google Scholar] [CrossRef] [PubMed]
- Phene, S.; Dunn, R.C.; Hammel, N.; Liu, Y.; Krause, J.; Kitade, N.; Schaekermann, M.; Sayres, R.; Wu, D.J.; Bora, A.; et al. Deep Learning and Glaucoma Specialists: The Relative Importance of Optic Disc Features to Predict Glaucoma Referral in Fundus Photographs. Ophthalmology 2019, 126, 1627–1639. [Google Scholar] [CrossRef]
- Maaliw, R.R.; Alon, A.S.; Lagman, A.C.; Garcia, M.B.; Abante, M.V.; Belleza, R.C.; Tan, J.B.; Maaño, R.A. Cataract Detection and Grading Using Ensemble Neural Networks and Transfer Learning. In Proceedings of the IEEE 13th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada, 12–15 October 2022; pp. 0074–0081. [Google Scholar] [CrossRef]
- Heisler, M.; Karst, S.; Lo, J.; Mammo, Z.; Yu, T.; Warner, S.; Maberley, D.; Beg, M.F.; Navajas, E.V.; Sarunic, M.V. Ensemble Deep Learning for Diabetic Retinopathy Detection Using Optical Coherence Tomography Angiography. Trans. Vis. Sci. Tech. 2020, 9, 20. [Google Scholar] [CrossRef]
- Lim, W.X.; Chen, Z.; Ahmed, A. The Adoption of Deep Learning Interpretability Techniques on Diabetic Retinopathy Analysis: A Review. Med. Biol. Eng. Comput. 2022, 60, 633–642. [Google Scholar] [CrossRef] [PubMed]
- Vasireddi, H.K.; Suganya Devi, K.; Raja Reddy, G.N.V. DR-XAI: Explainable Deep Learning Model for Accurate Diabetic Retinopathy Severity Assessment. Arab. J. Sci. Eng. 2024, 49, 12345–12356. [Google Scholar] [CrossRef]
- Mayya, V.; S, S.K.; Kulkarni, U.; Surya, D.K.; Acharya, U.R. An Empirical Study of Preprocessing Techniques with Convolutional Neural Networks for Accurate Detection of Chronic Ocular Diseases Using Fundus Images. Appl. Intell. 2023, 53, 1548–1566. [Google Scholar] [CrossRef]
- Zhao, Z.; Chen, H.; Wang, Y.-P.; Meng, D.; Xie, Q.; Yu, Q.; Wang, L. Retinal Disease Diagnosis with Unsupervised Grad-CAM Guided Contrastive Learning. Neurocomputing 2024, 593, 127816. [Google Scholar] [CrossRef]
- Saporta, A.; Gui, X.; Agrawal, A.; Pareek, A.; Truong, S.Q.H.; Nguyen, C.D.T.; Ngo, V.-D.; Seekins, J.; Blankenberg, F.G.; Ng, A.Y.; et al. Benchmarking Saliency Methods for Chest X-ray Interpretation. Nat. Mach. Intell. 2022, 4, 867–878. [Google Scholar] [CrossRef]
- Gao, Y.; Ventura-Diaz, S.; Wang, X.; He, M.; Xu, Z.; Weir, A.; Zhou, H.-Y.; Zhang, T.; van Duijnhoven, F.H.; Han, L.; et al. An Explainable Longitudinal Multi-Modal Fusion Model for Predicting Neoadjuvant Therapy Response in Women with Breast Cancer. Nat. Commun. 2024, 15, 9613. [Google Scholar] [CrossRef]
- Xu, Y.; Hosny, A.; Zeleznik, R.; Parmar, C.; Coroller, T.; Franco, I.; Mak, R.H.; Aerts, H.J.W.L. Deep Learning Predicts Lung Cancer Treatment Response from Serial Medical Imaging. Clin. Cancer Res. 2019, 25, 3266–3275. [Google Scholar] [CrossRef]
- Abdullakutty, F.; Akbari, Y.; Al-Maadeed, S.; Bouridane, A.; Talaat, I.M.; Hamoudi, R. Histopathology in Focus: A Review on Explainable Multi-Modal Approaches for Breast Cancer Diagnosis. Front. Med. 2024, 11, 1450103. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Z.; Deng, C.; Paulus, Y.M. Advances in Structural and Functional Retinal Imaging and Biomarkers for Early Detection of Diabetic Retinopathy. Biomedicines 2024, 12, 1405. [Google Scholar] [CrossRef]
- Hou, J.; Wang, L.L. Explainable AI for Clinical Outcome Prediction: A Survey of Clinician Perceptions and Preferences. AMIA Jt. Summits Transl. Sci. Proc. 2025, 2025, 215–224. [Google Scholar] [PubMed] [PubMed Central]
- Chattopadhyay, S. Decoding Medical Diagnosis with Machine Learning Classifiers. Medinformatics 2024. [CrossRef]
- Hashemi, M.; Darejeh, A.; Cruz, F. A User-Centric Exploration of Axiomatic Explainable AI in Participatory Budgeting. In Proceedings of the Companion of the 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Melbourne, Australia, 5–9 October 2024. [Google Scholar] [CrossRef]
- Sundararajan, M.; Taly, A.; Yan, Q. Axiomatic Attribution for Deep Networks. In Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017; Volume 70, pp. 3319–3328. Available online: https://proceedings.mlr.press/v70/sundararajan17a.html (accessed on 5 May 2025).
- Pawlicki, M.; Pawlicka, A.; Uccello, F.; Szelest, S.; D’Antonio, S.; Kozik, R.; Choraś, M. Evaluating the Necessity of the Multiple Metrics for Assessing Explainable AI. Neurocomputing 2024, 602, 128282. [Google Scholar] [CrossRef]
- Nauta, M.; Trienes, J.; Pathak, S.; Nguyen, E.; Peters, M.; Schmitt, Y.; Schlötterer, J.; van Keulen, M.; Seifert, C. From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI. arXiv 2023, arXiv:2201.08164. [Google Scholar] [CrossRef]
- Murdoch, W.J.; Singh, C.; Kumbier, K.; Abbasi-Asl, R.; Yu, B. Interpretable Machine Learning: Definitions, Methods, and Applications. arXiv 2019, arXiv:1901.04592. [Google Scholar] [CrossRef]
- Frangi, A.F.; Tsaftaris, S.A.; Prince, J.L. Simulation and Synthesis in Medical Imaging. IEEE Trans. Med. Imaging 2018, 37, 673–679. [Google Scholar] [CrossRef] [PubMed]
- Nguyen, G.; Kim, D.; Nguyen, A. The Effectiveness of Feature Attribution Methods and Its Correlation with Automatic Evaluation Scores. arXiv 2021, arXiv:2105.14944. Available online: https://arxiv.org/abs/2105.14944 (accessed on 5 May 2025).
- Carvalho, D.V.; Pereira, E.M.; Cardoso, J.S. Machine Learning Interpretability: A Survey on Methods and Metrics. Electronics 2019, 8, 832. [Google Scholar] [CrossRef]
Method Type | Key Techniques | Nature | Interpretability Characteristics |
---|---|---|---|
Gradient-Based | Saliency Maps, Integrated Gradients, SmoothGrad, Guided Backpropagation | Local, Post-hoc, Model-specific | Sensitive to gradients, fast, sometimes noisy |
Perturbation-Based | LIME, SHAP, Occlusion Sensitivity | Local, Post-hoc, Model-agnostic | Intuitive, costly (needs many forward passes), robust to noise |
CAM-Based | Grad-CAM, Grad-CAM++, Score-CAM, XGrad-CAM | Local, Post-hoc, Model-specific (CNNs) | Highlights class-specific regions, intuitive heatmaps |
Backpropagation-Based | LRP, DeepLIFT | Local, Post-hoc, Model-specific | Decomposes prediction into input relevance, rule-based redistribution |
Meta Attribution-Based | RISE, External Perturbations | Local, Post-hoc, Model-agnostic | Randomized sampling, useful when gradients are unavailable |
Modality | Attribution Method | Application | Performance | Ref. |
---|---|---|---|---|
X-ray/CT | Grad-CAM | COVID-19 Detection | Accuracy: 89.47–96.55% | [53] |
X-ray/CT | SHAP | Lung Cancer Detection | Enhanced Transparency | [54] |
MRI | LRP | Multiple Sclerosis Diagnosis | High Interpretability | [56] |
MRI | Grad-CAM | Brain Tumour Detection | High Accuracy | [57] |
CT | DeepLIFT | Liver Tumour Segmentation | Precise Attribution | [12] |
CT | Score-CAM | Lung Nodule Detection | High Specificity | [59] |
Modality | Attribution Method | Application | Performance | Ref. |
---|---|---|---|---|
Dermatoscopic | Grad-CAM | BCC Diagnosis | Accuracy: 90% | [61] |
Dermatoscopic | Grad-CAM | Skin Lesion Classification | Improved Detection | [62] |
Dermatoscopic | SmoothGrad, Score-CAM | Skin Cancer Categorisation | High Interpretability | [63,64] |
Dermatoscopic | XGrad-CAM | Melanoma Detection | Improved Localisation | [65] |
Modality | Attribution Method | Application | Performance | Ref. |
---|---|---|---|---|
Histopathology | Grad-CAM | Breast Cancer Classification | Accuracy: 94.3–98.7% | [67] |
Histopathology | Grad-CAM | Multi-Cancer Analysis | Biomarker Identification | [69] |
Histopathology | LIME, SHAP | Nasopharyngeal Cancer Survival | Feature Importance | [70] |
Histopathology | RISE | Breast Cancer Localisation | Complementary Attribution | [72] |
Modality | Attribution Method | Application | Performance | Ref. |
---|---|---|---|---|
Fundus | Grad-CAM | Glaucoma Detection | High Sensitivity and Specificity | [74] |
OCT | Integrated Gradients | Diabetic Retinopathy Detection | High Accuracy | [76] |
Fundus | Grad-CAM | Cataract Detection | Clinical Alignment | [75] |
Fundus | LIME | Retinal Disease Diagnosis | Finer-Grained Explanations | [78] |
Clinical Task | Attribution Method | Applications | Outcome | Ref. |
---|---|---|---|---|
Diagnosis and Classification | Grad-CAM, Saliency Map | COVID-19, Breast Cancer, BCC, Glaucoma, Tuberculosis | High Accuracy, Trust | [1,9,15,20,23,25] |
Lesion Detection and Segmentation | Grad-CAM, Integrated Gradients, Vanilla Backpropagation | Brain Tumours, Skin Lesions, Breast Cancer Lesions, Retinal Lesions | Improved Localisation | [5,10,16,22,26] |
Treatment Response Prediction | Grad-CAM, External Perturbation | Rectal Cancer, Breast Cancer, Skin Cancer | AUC: 0.95, Novel Insights | [10,15,27,28] |
Biomarker Discovery | Grad-CAM, LIME, SHAP, RISE | Retinal Patterns, Cancer Biomarkers | Potential Insights | [10,12,14,29] |
Clinical Requirement | Recommended Method | Rationale |
---|---|---|
High-resolution saliency needed | SmoothGrad | Reduces noise through input perturbation; highlights fine-grained anatomical features. |
Fast inference or real-time settings | Grad-CAM | Efficient and compatible with CNNs; improves localisation over standard rule-based explainers. |
Fair and complete attributions | Integrated Gradients | Offers completeness and sensitivity; strong for rigorous clinical interpretation. |
Gradient-free with visual clarity | Score-CAM | Bypasses backpropagation; produces sharp, noise-resistant saliency maps. |
Model-agnostic and consistent | SHAP | Based on Shapley values; ensures fairness and works across different model types. |
Black-box explanation | RISE | Requires no model internals; effective when transparency is limited. |
Simple for non-experts | LIME | Creates intuitive surrogate models; good for interdisciplinary use. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alam, K.N.; Zadeh, P.B.; Sheikh-Akbari, A. Attribution-Based Explainability in Medical Imaging: A Critical Review on Explainable Computer Vision (X-CV) Techniques and Their Applications in Medical AI. Electronics 2025, 14, 3024. https://doi.org/10.3390/electronics14153024
Alam KN, Zadeh PB, Sheikh-Akbari A. Attribution-Based Explainability in Medical Imaging: A Critical Review on Explainable Computer Vision (X-CV) Techniques and Their Applications in Medical AI. Electronics. 2025; 14(15):3024. https://doi.org/10.3390/electronics14153024
Chicago/Turabian StyleAlam, Kazi Nabiul, Pooneh Bagheri Zadeh, and Akbar Sheikh-Akbari. 2025. "Attribution-Based Explainability in Medical Imaging: A Critical Review on Explainable Computer Vision (X-CV) Techniques and Their Applications in Medical AI" Electronics 14, no. 15: 3024. https://doi.org/10.3390/electronics14153024
APA StyleAlam, K. N., Zadeh, P. B., & Sheikh-Akbari, A. (2025). Attribution-Based Explainability in Medical Imaging: A Critical Review on Explainable Computer Vision (X-CV) Techniques and Their Applications in Medical AI. Electronics, 14(15), 3024. https://doi.org/10.3390/electronics14153024