Saliency Map and Deep Learning in Binary Classification of Brain Tumours
Abstract
1. Introduction
1.1. Related Work
1.2. Contribution
1.3. Framework of Research
1.4. Structure
2. Methods
2.1. Architectures of Deep Neural Networks
- input layer, which processes the spatial data of the image;
- feature-extracting layers—they are arranged in a general sequence containing a convolutional layer that uses numerous filters to learn various features of data received from the input layer (the obtained result is transformed using the ReLU activation function), and a pooling layer (the task is to gradually reduce the spatial size of the data representation);
- classification layers or output layer (in most cases, it is a fully connected layer) used to compute class scores as a result of network operation.
2.2. Saliency Maps
2.2.1. Class Activation Mapping
2.2.2. Grad-CAM Method
3. Results of Experiments and Discussion
3.1. Metrics
3.2. Comparison of Convolutional Networks
3.3. Results Obtained by the CAM and Grad-CAM Methods
- -
- denotes the percentage average value of the difference between the average Cartesian distance of CoM for the CAM and Grad-CAM methods,
- -
- denotes the percentage average value of the difference between the average IoU value for the CAM and Grad-CAM methods.
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
CAM | Class Activation Mapping |
Grad-CAM | Gradient-based Class Activation Mapping |
CNN | Convolutional Neural Network |
IoU | Intersection over Union |
CoM | Center of Mass |
ResNet | Residual Network |
VGG | Visual Geometry Group |
MRI | Magnetic Resonance Image |
CaD | Cartesian Distance of CoM |
References
- McFaline-Figueroa, J.R.; Lee, E.Q. Brain tumors. Am. J. Med. 2018, 131, 874–882. [Google Scholar] [CrossRef] [PubMed]
- White, B.J.; Berg, D.J.; Kan, J.Y.; Marino, R.A.; Itti, L.; Munoz, D.P. Superior colliculus neurons encode a visual saliency map during free viewing of natural dynamic video. Nat. Commun. 2017, 8, 14263. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Saporta, A.; Gui, X.; Agrawal, A.; Pareek, A.; Truong, S.Q.H.; Nguyen, C.D.T.; Ngo, V.-D.; Seekins, J.; Blankenberg, F.G.; Ng, A.Y.; et al. Benchmarking saliency methods for chest X-ray interpretation. Nat. Mach. Intell. 2022, 4, 867–878. [Google Scholar] [CrossRef]
- Wang, Z.J. Probing an AI regression model for hand bone age determination using gradient-based saliency mapping. Sci. Rep. 2021, 11, 10610. [Google Scholar] [CrossRef] [PubMed]
- Ghosh, S.; D’Angelo, G.; Glover, A.; Iacono, M.; Niebur, E.; Bartolozzi, C. Event-driven proto-object based saliency in 3D space to attract a robot’s attention. Sci. Rep. 2022, 12, 7645. [Google Scholar] [CrossRef]
- Amorim, J.P.; Abreu, P.H.; Santos, J.; Cortes, M.; Vila, V. Evaluating the faithfulness of saliency maps in explaining deep learning models using realistic perturbations. Inf. Process. Manag. 2023, 60, 103225. [Google Scholar] [CrossRef]
- Ayhan, M.S.; Kümmerle, L.B.; Kühlewein, L.; Inhoffen, W.; Aliyeva, G.; Ziemssen, F.; Berens, P. Clinical validation of saliency maps for understanding deep neural networks in ophthalmology. Med. Image Anal. 2022, 77, 102364. [Google Scholar] [CrossRef] [PubMed]
- Saeedi, S.; Rezayi, S.; Keshavarz, H.; Kalhori, S.R.N. MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques. BMC Med. Inform. Decis. Mak. 2023, 23, 16. [Google Scholar] [CrossRef] [PubMed]
- Humayun, M.; Khalil, M.I.; Alwakid, G.; Jhanjhi, N.Z. Superlative Feature Selection Based Image Classification Using Deep Learning in Medical Imaging. J. Healthc. Eng. 2020, 2022, 7028717. [Google Scholar] [CrossRef] [PubMed]
- Saida, D.; Premchand, P. Brain Tumor Identification using Dilated U-Net based CNN. Int. J. Comput. Commun. Control 2022, 17. [Google Scholar] [CrossRef]
- Khan, M.A.; Ashraf, I.; Alhaisoni, M.; Damaševicius, R.; Scherer, R.; Rehman, A.; Bukhari, S.A.C. Multimodal brain tumor classification using deep learning and robust feature selection: A machine learning application for radiologists. Diagnostics 2020, 10, 565. [Google Scholar] [CrossRef] [PubMed]
- Amin, J.; Sharif, M.; Gul, N.; Yasmin, M.; Shad, S.A. Brain tumor classification based on DWT fusion of MRI sequences using convolutional neural network. Pattern Recognit. Lett. 2020, 129, 115–122. [Google Scholar] [CrossRef]
- Sajjad, M.; Khan, S.; Muhammad, K.; Wu, W.; Ullah, A.; Baik, S.W. Multi-grade brain tumor classification using deep CNN with extensive data augmentation. J. Comput. Sci. 2019, 30, 174–182. [Google Scholar] [CrossRef]
- Kokkalla, S.; Kakarla, J.; Venkateswarlu, I.B.; Singh, M. Three-class brain tumor classification using deep dense inception residual network. Soft Comput. 2021, 25, 8721–8729. [Google Scholar] [CrossRef] [PubMed]
- Alsubai, S.; Khan, H.U.; Alqahtani, A.; Sha, M.; Abbas, S.; Mohammad, U.G. Ensemble deep learning for brain tumor detection. Front. Comput. Neurosci. 2022, 16, 1005617. [Google Scholar] [CrossRef] [PubMed]
- Brain Tumor Dataset. Available online: https://www.kaggle.com/datasets/preetviradiya/brian-tumor-dataset (accessed on 11 October 2022).
- Varga, D. No-Reference Image Quality Assessment with Convolutional Neural Networks and Decision Fusion. Appl. Sci 2022, 12, 101. [Google Scholar] [CrossRef]
- Simonyan., K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Tan, M.; Le, Q.V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA, 9–15 June 2019. [Google Scholar]
- Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning Deep Features for Discriminative Localization. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 2921–2929. [Google Scholar]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Int. J. Comput. Vis. 2020, 128, 336–359. [Google Scholar] [CrossRef][Green Version]
- Zhang, L.; Shen, Y.; Li, H. VSI: A visual saliency induced index for perceptual image quality assessment. IEEE Trans. Image Process. 2014, 128, 4270–4281. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Zhou, W.; Yue, G.; Zhang, R.; Qin, Y.; Li, H. Reduced-reference quality assessment of point clouds via content-oriented saliency projection. IEEE Signal Process. Lett. 2023, 30, 354–358. [Google Scholar] [CrossRef]
- Rezatofighi, H.; Tsoi, N.; Gwak, J.; Sadeghian, A.; Reid, I.; Savarese, S. Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019; pp. 658–666. [Google Scholar] [CrossRef][Green Version]
- Chattopadhay, A.; Sarkar, A.; Howlader, P.; Balasubramanian, V.N. Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks. In Proceeding of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA, 12–15 March 2018; pp. 839–847. [Google Scholar]
- Bany Muhammad, M.; Yeasin, M. Eigen-CAM: Visual explanations for deep convolutional neural networks. SN Comput. Sci. 2021, 2, 336–359. [Google Scholar] [CrossRef]
Architecture | Layers Number | Total Parameters | Trainable Parameters | Train Accuracy | Test Accuracy | Recall | Precision |
---|---|---|---|---|---|---|---|
VGG16 | 16 + 3 | 14,978,370 | 263,682 | ||||
ResNet50 | 50 + 3 | 25,947,394 | 2,359,682 | ||||
EfficientNet | 813 + 3 | 67,047,193 | 2,949,506 | ||||
CNN | 10 | 97,458 | 97,458 |
Intersection over Union (IoU) | Cartesian Distance (CaD) of CoM | |||||
---|---|---|---|---|---|---|
CAM | Grad-CAM | (%) | CAM | Grad-CAM | (%) | |
CNN + EfficientNet | 0.635 | 0.627 | 1.26 | 0.2121 | 0.2496 | −17.72 |
CNN + VGG | 0.447 | 0.562 | −25.73 | 0.2893 | 0.2913 | −0.69 |
CNN + ResNet | 0.618 | 0.602 | 2.59 | 0.2229 | 0.2626 | −17.78 |
ResNet + EfficientNet | 0.854 | 0.851 | 0.35 | 0.0916 | 0.1208 | −31.92 |
ResNet + VGG | 0.572 | 0.699 | −22.20 | 0.2396 | 0.1398 | 41.65 |
VGG + EfficientNet | 0.579 | 0.711 | −22.80 | 0.2156 | 0.1067 | 50.50 |
Avr | 0.618 | 0.675 | 0.2118 | 0.1951 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chmiel, W.; Kwiecień, J.; Motyka, K. Saliency Map and Deep Learning in Binary Classification of Brain Tumours. Sensors 2023, 23, 4543. https://doi.org/10.3390/s23094543
Chmiel W, Kwiecień J, Motyka K. Saliency Map and Deep Learning in Binary Classification of Brain Tumours. Sensors. 2023; 23(9):4543. https://doi.org/10.3390/s23094543
Chicago/Turabian StyleChmiel, Wojciech, Joanna Kwiecień, and Kacper Motyka. 2023. "Saliency Map and Deep Learning in Binary Classification of Brain Tumours" Sensors 23, no. 9: 4543. https://doi.org/10.3390/s23094543
APA StyleChmiel, W., Kwiecień, J., & Motyka, K. (2023). Saliency Map and Deep Learning in Binary Classification of Brain Tumours. Sensors, 23(9), 4543. https://doi.org/10.3390/s23094543