An Effective Skin Cancer Classification Mechanism via Medical Vision Transformer
Abstract
:1. Introduction
- The major problem in the field of SC is the lack of publicly available datasets. Although there are some datasets available on the internet with a limited number of samples, these datasets are imbalanced, which negatively affects the performance of a model Thus, we used extensive data augmentation by augmenting the data with various parameters and different techniques to fill the data gap and make the system transformational and noise invariant.
- The current literature focuses on CNN-based models for SC classification, but these models reduce the dimensionality of the input data, which causes loss of meaningful information. To fill this gap we developed a vision transformer-based model with the potential of learning from a whole set of features, providing better accuracy.
- We evaluated the proposed model on a publicly available HAM10000 dataset and obtained higher performance in terms of F1-score, specificity, sensitivity, and accuracy when compared to state-of-the-art methods.
2. The Proposed Method
2.1. Data Preprocessing
2.2. Our Proposed MVT Model for SC Classification
Algorithm 1 Training and testing of the proposed MVT model |
Input: Dataset of SC images Dataset split: Training, Validation and Testing |
Output: predicted labels |
Training: |
|
|
|
|
Model testing: |
|
|
3. Experimental Setup and Results
3.1. Results Evaluation with and without Preprocessing
3.2. Comparison of the Proposed Method with State-of-the-Art Methods
4. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Bibi, A.; Khan, M.A.; Javed, M.Y.; Tariq, U.; Kang, B.-G.; Nam, Y.; Mostafa, R.R.; Sakr, R.H. Skin Lesion Segmentation and Classification Using Conventional and Deep Learning Based Framework. Comput. Mater. Contin. 2022, 71, 2477–2495. [Google Scholar] [CrossRef]
- Baig, R.; Bibi, M.; Hamid, A.; Kausar, S.; Khalid, S. Deep Learning Approaches Towards Skin Lesion Segmentation and Classification from Dermoscopic Images—A Review. Curr. Med Imaging 2020, 16, 513–533. [Google Scholar] [CrossRef] [PubMed]
- Razzak, I.; Naz, S. Unit-Vise: Deep Shallow Unit-Vise Residual Neural Networks With Transition Layer For Expert Level Skin Cancer Classification. IEEE/ACM Trans. Comput. Biol. Bioinform. 2020, 19, 1225–1234. [Google Scholar] [CrossRef] [PubMed]
- Parshionikar, S.; Koshy, R.; Sheikh, A.; Phansalkar, G. Skin Cancer Detection and Severity Prediction Using Computer Vision and Deep Learning. In Proceedings of the Second International Conference on Sustainable Technologies for Computational Intelligence, Dehradun, India, 23–24 December 2022; Springer: New York, NY, USA, 2022; pp. 295–304. [Google Scholar]
- Campanella, G.; Navarrete-Dechent, C.; Liopyris, K.; Monnier, J.; Aleissa, S.; Minhas, B.; Scope, A.; Longo, C.; Guitera, P.; Pellacani, G.; et al. Deep Learning for Basal Cell Carcinoma Detection for Reflectance Confocal Microscopy. J. Investig. Dermatol. 2021, 142, 97–103. [Google Scholar] [CrossRef]
- Singh, S.K.; Jalal, A.S. A robust approach for automatic skin cancer disease classification. In Proceedings of the 1st India International Conference on Information Processing (IICIP), Delhi, India, 12–14 August 2016; pp. 1–4. [Google Scholar]
- Ansari, U.B.; Sarode, T. Skin cancer detection using image processing. Int. Res. J. Eng. Technol. 2017, 4, 2875–2881. [Google Scholar]
- Taufiq, M.A.; Hameed, N.; Anjum, A.; Hameed, F. m-Skin Doctor: A Mobile Enabled System for Early Melanoma Skin Cancer Detection Using Support Vector Machine. eHealth 2016, 181, 468–475. [Google Scholar] [CrossRef]
- Alfed, N.; Khelifi, F. Bagged textural and color features for melanoma skin cancer detection in dermoscopic and standard images. Expert Syst. Appl. 2017, 90, 101–110. [Google Scholar] [CrossRef] [Green Version]
- Alquran, H.; Oasmieh, I.; Algudah, A.M.; Alhammouri, S.; Alawneh, E.; Abughazaleh, A.; Hasayen, F. The melanoma skin cancer detection and classification using support vector machine. In Proceedings of the IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT), Aqaba, Jordan, 11–13 October 2017; pp. 1–5. [Google Scholar]
- Thanh, D.N.; Prasath, V.; Hieu, L.M.; Hien, N.N. Melanoma skin cancer detection method based on adaptive principal curvature, colour normalisation and feature extraction with the ABCD rule. J. Digit. Imaging 2020, 33, 574–585. [Google Scholar] [CrossRef]
- Victor, A.; Ghalib, M. Automatic detection and classification of skin cancer. Int. J. Intell. Eng. Syst. 2017, 10, 444–451. [Google Scholar] [CrossRef]
- Celebi, M.E.; Codella, N.; Halpern, A. Dermoscopy Image Analysis: Overview and Future Directions. IEEE J. Biomed. Health Inform. 2019, 23, 474–478. [Google Scholar] [CrossRef]
- Javed, R.; Rahim, M.S.M.; Saba, T.; Fati, S.M.; Rehman, A.; Tariq, U. Statistical Histogram Decision Based Contrast Categorization of Skin Lesion Datasets Dermoscopic Images. Comput. Mater. Contin. 2021, 67, 2337–2352. [Google Scholar] [CrossRef]
- Yar, H.; Hussain, T.; Khan, Z.A.; Koundal, D.; Lee, M.Y.; Baik, S.W. Vision Sensor-Based Real-Time Fire Detection in Resource-Constrained IoT Environments. Comput. Intell. Neurosci. 2021, 2021, 5195508. [Google Scholar] [CrossRef] [PubMed]
- Shakhnoza, M.; Sabina, U.; Sevara, M.; Cho, Y.-I. Novel Video Surveillance-Based Fire and Smoke Classification Using Attentional Feature Map in Capsule Networks. Sensors 2021, 22, 98. [Google Scholar] [CrossRef]
- Ali, H.; Farman, H.; Yar, H.; Khan, Z.; Habib, S.; Ammar, A. Deep learning-based election results prediction using Twitter activity. Soft Comput. 2021, 1–9. [Google Scholar] [CrossRef]
- Yar, H.; Abbas, N.; Sadad, T.; Iqbal, S. Lung Nodule Detection and Classification Using 2D and 3D Convolution Neural Networks (CNNs); CRC Press: Boca Raton, FL, USA, 2021; pp. 365–386. [Google Scholar] [CrossRef]
- Norouzi, A.; Rahinm, M.S.M.; Saba, T.; Altameem, A. Medical image segmentation methods, algorithms, and applications. IETE Tech. Rev. 2014, 31, 199–213. [Google Scholar] [CrossRef]
- Muhammad, K.; Ullah, H.; Khan, Z.A.; Saudagar, A.K.J.; AlTameem, A.; AlKhathami, M.; Khan, M.B.; Hasanat, M.H.A.; Malik, K.M.; Hijji, M.; et al. WEENet: An Intelligent System for Diagnosing COVID-19 and Lung Cancer in IoMT Environments. Front. Oncol. 2021, 11, 811355. [Google Scholar] [CrossRef]
- Khan, Z.A.; Hussain, T.; Baik, S.W. Boosting energy harvesting via deep learning-based renewable power generation prediction. J. King Saud Univ. Sci. 2021, 34, 101815. [Google Scholar] [CrossRef]
- Khan, Z.A.; Ullah, A.; Ullah, W.; Rho, S.; Lee, M.; Baik, S.W. Electrical Energy Prediction in Residential Buildings for Short-Term Horizons Using Hybrid Deep Learning Strategy. Appl. Sci. 2020, 10, 8634. [Google Scholar] [CrossRef]
- Khan, Z.; Hussain, T.; Ullah, A.; Rho, S.; Lee, M.; Baik, S. Towards Efficient Electricity Forecasting in Residential and Commercial Buildings: A Novel Hybrid CNN with a LSTM-AE based Framework. Sensors 2020, 20, 1399. [Google Scholar] [CrossRef] [Green Version]
- Khan, S.U.; Haq, I.U.; Khan, Z.A.; Khan, N.; Lee, M.Y.; Baik, S.W. Atrous Convolutions and Residual GRU Based Architecture for Matching Power Demand with Supply. Sensors 2021, 21, 7191. [Google Scholar] [CrossRef]
- Sajjad, M.; Khan, Z.A.; Ullah, A.; Hussain, T.; Ullah, W.; Lee, M.Y.; Baik, S.W. A Novel CNN-GRU-Based Hybrid Approach for Short-Term Residential Load Forecasting. IEEE Access 2020, 8, 143759–143768. [Google Scholar] [CrossRef]
- Khan, S.U.; Hussain, T.; Ullah, A.; Baik, S.W. Deep-ReID: Deep features and autoencoder assisted image patching strategy for person re-identification in smart cities surveillance. In Multimedia Tools and Applications; Springer: New York, NY, USA, 2021; pp. 1–22. [Google Scholar]
- Khan, Z.A.; Ullah, W.; Ullah, A.; Rho, S.; Lee, M.Y.; Baik, S.W. An Adaptive Filtering Technique for Segmentation of Tuberculosis in Microscopic Images. In Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval, Seoul, Korea, 18–20 December 2020. [Google Scholar] [CrossRef]
- Adegun, A.; Viriri, S. Deep learning techniques for skin lesion analysis and melanoma cancer detection: A survey of state-of-the-art. Artif. Intell. Rev. 2020, 54, 811–841. [Google Scholar] [CrossRef]
- Javed, R.; Rahim, M.S.M.; Saba, T.; Rehman, A. A comparative study of features selection for skin lesion detection from dermoscopic images. Netw. Model. Anal. Health Inform. Bioinform. 2019, 9, 4. [Google Scholar] [CrossRef]
- Saba, T.; Khan, M.A.; Rehman, A.; Marie-Sainte, S.L. Region Extraction and Classification of Skin Cancer: A Heterogeneous framework of Deep CNN Features Fusion and Reduction. J. Med Syst. 2019, 43, 289. [Google Scholar] [CrossRef]
- Haggenmüller, S.; Maron, R.C.; Hekler, A.; Utikal, J.S.; Barata, C.; Barnhill, R.L.; Beltraminelli, H.; Berking, C.; Betz-Stablein, B.; Blum, A.; et al. Skin cancer classification via convolutional neural networks: Systematic review of studies involving human experts. Eur. J. Cancer 2021, 156, 202–216. [Google Scholar] [CrossRef]
- Araújo, R.L.; Ricardo de Andrade, L.R.; Rodrigues, J.J.; e Silva, R.R. Automatic Segmentation of Melanoma Skin Cancer Using Deep Learning. In Proceedings of the IEEE International Conference on E-health Networking, Application & Services (HEALTHCOM), Shenzhen, China, 1–2 March 2021; pp. 1–6. [Google Scholar]
- Khamparia, A.; Singh, P.K.; Rani, P.; Samanta, D.; Khanna, A.; Bhushan, B. An internet of health things-driven deep learning framework for detection and classification of skin cancer using transfer learning. Trans. Emerg. Telecommun. Technol. 2021, 32, e3963. [Google Scholar] [CrossRef]
- Khan, M.A.; Sharif, M.I.; Raza, M.; Anjum, A.; Saba, T.; Shad, S.A. Skin lesion segmentation and classification: A unified framework of deep neural network features fusion and selection. Expert Syst. 2019, e12497. [Google Scholar] [CrossRef]
- Ding, S.; Huang, H.; Li, Z.; Liu, X.; Yang, S. SCNET: A Novel UGI Cancer Screening Framework Based on Semantic-Level Multimodal Data Fusion. IEEE J. Biomed. Health Inform. 2020, 25, 143–151. [Google Scholar] [CrossRef]
- Khan, M.A.; Akram, T.; Sharif, M.; Javed, K.; Rashid, M.; Bukhari, S.A.C. An integrated framework of skin lesion detection and recognition through saliency method and optimal deep neural network features selection. Neural Comput. Appl. 2019, 32, 15929–15948. [Google Scholar] [CrossRef]
- Huang, H.; Hsu, B.W.; Lee, C.; Tseng, V.S. Development of a light-weight deep learning model for cloud applications and remote diagnosis of skin cancers. J. Dermatol. 2020, 48, 310–316. [Google Scholar] [CrossRef]
- Carcagnì, P.; Leo, M.; Cuna, A.; Mazzeo, P.L.; Spagnolo, P.; Celeste, G.; Distante, C. Classification of Skin Lesions by Combining Multilevel Learnings in a DenseNet Architecture. In Proceedings of the 20th International Conference Image Analysis and Processing (ICIAP 2019), Trento, Italy, 9–13 September 2019; Springer: New York, NY, USA, 2019; pp. 335–344. [Google Scholar] [CrossRef]
- Thurnhofer-Hemsi, K.; Domínguez, E. A Convolutional Neural Network Framework for Accurate Skin Cancer Detection. Neural Process. Lett. 2020, 53, 3073–3093. [Google Scholar] [CrossRef]
- Mohamed, E.H.; El-Behaidy, W.H. Enhanced skin lesions classification using deep convolutional networks. In Proceedings of the Ninth International Conference on Intelligent Computing and Information Systems (ICICIS), Cairo, Egypt, 8–10 December 2019; pp. 180–188. [Google Scholar]
- Chaturvedi, S.S.; Tembhurne, J.V.; Diwan, T. A multi-class skin Cancer classification using deep convolutional neural networks. Multimed. Tools Appl. 2020, 79, 28477–28498. [Google Scholar] [CrossRef]
- Shahin, A.H.; Kamal, A.; Elattar, M.A. Deep Ensemble Learning for Skin Lesion Classification from Dermoscopic Images. In Proceedings of the 9th Cairo International Biomedical Engineering Conference (CIBEC), Cairo, Egypt,, 20–22 December 2018; pp. 150–153. [Google Scholar] [CrossRef]
- Almaraz-Damian, J.-A.; Ponomaryov, V.; Sadovnychiy, S.; Castillejos-Fernandez, H. Melanoma and Nevus Skin Lesion Classification Using Handcraft and Deep Learning Feature Fusion via Mutual Information Measures. Entropy 2020, 22, 484. [Google Scholar] [CrossRef] [Green Version]
- Zhang, J.; Xie, Y.; Xia, Y.; Shen, C. Attention Residual Learning for Skin Lesion Classification. IEEE Trans. Med. Imaging 2019, 38, 2092–2103. [Google Scholar] [CrossRef] [PubMed]
- Agrahari, P.; Agrawal, A.; Subhashini, N. Skin Cancer Detection Using Deep Learning. In Futuristic Communication and Network Technologies; Springer: New York, NY, USA, 2022; pp. 179–190. [Google Scholar]
- Jain, S.; Singhania, U.; Tripathy, B.; Nasr, E.A.; Aboudaif, M.K.; Kamrani, A.K. Deep Learning-Based Transfer Learning for Classification of Skin Cancer. Sensors 2021, 21, 8142. [Google Scholar] [CrossRef]
- Nawaz, M.; Mehmood, Z.; Nazir, T.; Naqvi, R.A.; Rehman, A.; Iqbal, M.; Saba, T. Skin cancer detection from dermoscopic images using deep learning and fuzzy k-means clustering. Microsc. Res. Tech. 2021, 85, 339–351. [Google Scholar] [CrossRef]
- Sharma, A.K.; Tiwari, S.; Aggarwal, G.; Goenka, N.; Kumar, A.; Chakrabarti, P.; Chakrabarti, T.; Gono, R.; Leonowicz, Z.; Jasinski, M. Dermatologist-Level Classification of Skin Cancer Using Cascaded Ensembling of Convolutional Neural Network and Handcrafted Features Based Deep Neural Network. IEEE Access 2022, 10, 17920–17932. [Google Scholar] [CrossRef]
- Shahid, M.; Hua, K.-L. Fire detection using transformer network. In Proceedings of the International Conference on Multimedia Retrieval, Newark, NJ, USA, 27–30 June 2021; pp. 627–630. [Google Scholar]
- Yar, H.; Hussain, T.; Khan, Z.A.; Lee, M.Y.; Baik, S.-W. Fire Detection via Effective Vision Transformers. J. KINGComputing 2021, 17, 21–30. [Google Scholar]
- Yuan, H.; Cai, Z.; Zhou, H.; Wang, Y.; Chen, X. TransAnomaly: Video Anomaly Detection Using Video Vision Transformer. IEEE Access 2021, 9, 123977–123986. [Google Scholar] [CrossRef]
- Dai, Y.; Gao, Y.; Liu, F. TransMed: Transformers Advance Multi-Modal Medical Image Classification. Diagnostics 2021, 11, 1384. [Google Scholar] [CrossRef]
- Chen, J.; He, Y.; Frey, E.C.; Li, Y.; Du, Y. ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration. arXiv 2021, arXiv:2104.06468. [Google Scholar]
- Habib, S.; Hussain, A.; Albattah, W.; Islam, M.; Khan, S.; Khan, R.U.; Khan, K. Abnormal Activity Recognition from Surveillance Videos Using Convolutional Neural Network. Sensors 2021, 21, 8291. [Google Scholar] [CrossRef] [PubMed]
- Habib, S.; Alsanea, M.; Aloraini, M.; Al-Rawashdeh, H.S.; Islam, M.; Khan, S. An Efficient and Effective Deep Learning-Based Model for Real-Time Face Mask Detection. Sensors 2022, 22, 2602. [Google Scholar] [CrossRef] [PubMed]
- Jan, H.; Yar, H.; Iqbal, J.; Farman, H.; Khan, Z.; Koubaa, A. Raspberry Pi Assisted Safety System for Elderly People: An Application of Smart Home. In Proceedings of the First International Conference of Smart Systems and Emerging Technologies (SMARTTECH), Riyagh, Saudi Arabia, 3–5 November 2020; pp. 155–160. [Google Scholar]
- Yar, H.; Imran, A.S.; Khan, Z.A.; Sajjad, M.; Kastrati, Z.J.S. Towards smart home automation using IoT-enabled edge-computing paradigm. Sensors 2021, 21, 4932. [Google Scholar] [CrossRef]
- Ullah, W.; Ullah, A.; Hussain, T.; Muhammad, K.; Heidari, A.A.; Del Ser, J.; Baik, S.W.; De Albuquerque, V.H.C. Artificial Intelligence of Things-assisted two-stream neural network for anomaly detection in surveillance Big Video Data. Futur. Gener. Comput. Syst. 2021, 129, 286–297. [Google Scholar] [CrossRef]
- Alsunaidi, S.; Almuhaideb, A.; Ibrahim, N.; Shaikh, F.; Alqudaihi, K.; Alhaidari, F.; Khan, I.; Aslam, N.; Alshahrani, M. Applications of Big Data Analytics to Control COVID-19 Pandemic. Sensors 2021, 21, 2282. [Google Scholar] [CrossRef]
- Chaturvedi, S.S.; Gupta, K.; Prasad, P.S. Skin Lesion Analyser: An Efficient Seven-Way Multi-class Skin Cancer Classification Using MobileNet. In Proceedings of the International Conference on Advanced Machine Learning Technologies and Applications, Jaipur, India, 13–15 February 2020; Springer: New York, NY, USA, 2020; pp. 165–176. [Google Scholar]
No. | Class | Number of Images | Number of Images |
---|---|---|---|
Before Preprocessing | After Preprocessing | ||
1 | Akiec | 327 | 1099 |
2 | Bcc | 541 | 1099 |
3 | Bkl | 1099 | 1099 |
4 | Df | 155 | 1099 |
5 | Nv | 6705 | 6705 |
6 | Mel | 1113 | 1113 |
7 | Vasc | 142 | 1099 |
Class | Akiec | Bcc | Bkl | Df | Mel | Nv | Vasc | Class-Wise Accuracy |
---|---|---|---|---|---|---|---|---|
Akiec | 0.97 | 0.02 | 0.0 | 0.0 | 0.0 | 0.01 | 0.0 | 97.00% |
Bcc | 0.09 | 0.81 | 0.07 | 0.0 | 0.02 | 0.00 | 0.01 | 81.00% |
Bkl | 0.02 | 0.02 | 0.83 | 0.0 | 0.05 | 0.08 | 0.0 | 83.00% |
Df | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 100% |
Mel | 0.02 | 0.0 | 0.08 | 0.0 | 0.78 | 0.12 | 0.0 | 78.00% |
Nv | 0.0 | 0.0 | 0.02 | 0.0 | 0.04 | 0.93 | 0.01 | 93.00% |
Vasc | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 100% |
Class | Akiec | Bcc | Bkl | Df | Mel | Nv | Vasc | Class-Wise Accuracy |
---|---|---|---|---|---|---|---|---|
Akiec | 0.96 | 0.01 | 0.0 | 0.0 | 0.0 | 0.02 | 0.1 | 96.00% |
Bcc | 0.01 | 0.91 | 0.04 | 0.01 | 0.02 | 0.01 | 0.0 | 91.00% |
Bkl | 0.02 | 0.02 | 0.94 | 0.0 | 0.01 | 0.01 | 0.0 | 94.00% |
Df | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 100% |
Mel | 0.0 | 0.0 | 0.03 | 0.0 | 0.95 | 0.02 | 0.0 | 95.00% |
Nv | 0.0 | 0.0 | 0.0 | 0.0 | 0.02 | 0.97 | 0.01 | 97.00% |
Vasc | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 100% |
Reference | Precision | Recall/ Sensitivity | F1-Measure | Accuracy |
---|---|---|---|---|
Attique et al. [59] | 92.22 | 84.20 | 88.03 | 95.80 |
Gupta et al. [60] | 89.00 | 83.00 | 83.00 | 83.10 |
Chaturvedi et al. [41] | 88.00 | 88.00 | 88.00 | 93.20 |
Huang et al. [37] | 75.18 | -- | -- | 85.80 |
Carcagni et al. [38] | 88.00 | 76.00 | 82.00 | 90.00 |
Shahin et al. [42] | 86.20 | 79.60 | 82.90 | 89.90 |
Jain et al. [46] | 88.76 | 89.57 | 89.02 | 90.48 |
The proposed method | 96.00 | 96.50 | 97.00 | 96.14 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Aladhadh, S.; Alsanea, M.; Aloraini, M.; Khan, T.; Habib, S.; Islam, M. An Effective Skin Cancer Classification Mechanism via Medical Vision Transformer. Sensors 2022, 22, 4008. https://doi.org/10.3390/s22114008
Aladhadh S, Alsanea M, Aloraini M, Khan T, Habib S, Islam M. An Effective Skin Cancer Classification Mechanism via Medical Vision Transformer. Sensors. 2022; 22(11):4008. https://doi.org/10.3390/s22114008
Chicago/Turabian StyleAladhadh, Suliman, Majed Alsanea, Mohammed Aloraini, Taimoor Khan, Shabana Habib, and Muhammad Islam. 2022. "An Effective Skin Cancer Classification Mechanism via Medical Vision Transformer" Sensors 22, no. 11: 4008. https://doi.org/10.3390/s22114008
APA StyleAladhadh, S., Alsanea, M., Aloraini, M., Khan, T., Habib, S., & Islam, M. (2022). An Effective Skin Cancer Classification Mechanism via Medical Vision Transformer. Sensors, 22(11), 4008. https://doi.org/10.3390/s22114008