Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence

Despite significant advancements in the automatic classification of skin lesions using artificial intelligence (AI) algorithms, skepticism among physicians persists. This reluctance is primarily due to the lack of transparency and explainability inherent in these models, which hinders their widespread acceptance in clinical settings. The primary objective of this study is to develop a highly accurate AI-based algorithm for skin lesion classification that also provides visual explanations to foster trust and confidence in these novel diagnostic tools. By improving transparency, the study seeks to contribute to earlier and more reliable diagnoses. Additionally, the research investigates the impact of Test Time Augmentation (TTA) on the performance of six Convolutional Neural Network (CNN) architectures, which include models from the EfficientNet, ResNet (Residual Network), and ResNeXt (an enhanced variant of ResNet) families. To improve the interpretability of the models’ decision-making processes, techniques such as t-distributed Stochastic Neighbor Embedding (t-SNE) and Gradient-weighted Class Activation Mapping (Grad-CAM) are employed. t-SNE is utilized to visualize the high-dimensional latent features of the CNNs in a two-dimensional space, providing insights into how the models group different skin lesion classes. Grad-CAM is used to generate heatmaps that highlight the regions of input images that influence the model’s predictions. Our findings reveal that Test Time Augmentation enhances the balanced multi-class accuracy of CNN models by up to 0.3%, achieving a balanced accuracy rate of 97.58% on the International Skin Imaging Collaboration (ISIC 2019) dataset. This performance is comparable to, or marginally better than, more complex approaches such as Vision Transformers (ViTs), demonstrating the efficacy of our methodology.