4.1. Classification Results
The classification performance of the fine-tuned ResNet50 and DenseNet121 models was evaluated using the BRISC validation subset.
Figure 3 show the training and validation loss curves, while
Figure 4 present the corresponding validation accuracy and F1-score curves.
Figure 5 shows the training and validation loss curves for the DenseNet121 model.
Figure 6 presents the corresponding validation accuracy and F1-score curves.
ResNet50 achieved a validation accuracy of 99.1% after 18 training epochs, whereas DenseNet121 achieved 99.2% after 19 epochs. The F1-score followed the same trend as the validation accuracy for both models throughout training.
The high classification performance of both CNN architectures allowed them to be used for the explainability experiments described in the following sections.
Both CNN architectures achieved high classification accuracy on the BRISC dataset. ResNet50 achieved a maximum validation accuracy of 99.1%, whereas DenseNet121 achieved 99.2%. As both models performed similarly, they were used in the subsequent explainability experiments.
4.2. Cross-Validation Analysis
To assess the stability of the proposed framework, an additional five-fold cross-validation experiment was performed for both CNN architectures using the complete preprocessing pipeline. The same training settings were used for all folds. The average classification results are summarised in
Table 2, while the distributions of validation accuracy and F1-score are shown in
Figure 7,
Figure 8,
Figure 9 and
Figure 10.
The results were similar across all five folds for both CNN architectures and validation accuracy, and F1-score showed only small variations between the individual folds, indicating stable model performance throughout the cross-validation experiments.
Figure 7,
Figure 8,
Figure 9 and
Figure 10 illustrate the distribution of the classification metrics for each model. The relatively small spread of the boxplots indicates that the obtained results were not dominated by a particular train–validation split.
Overall, the cross-validation results show that both CNN architectures maintain stable classification performance across different data partitions.
The five-fold cross-validation produced similar results for both CNN architectures. Validation accuracy and F1-score showed only small differences between the individual folds.
Figure 7,
Figure 8,
Figure 9 and
Figure 10 summarise the distribution of these metrics. The narrow spread of the boxplots indicates that the models performed consistently across all folds.
Overall, the cross-validation results suggest that the reported classification performance is not dependent on a particular train–validation split.
4.3. Explainability Results
The explainability performance of the investigated CNN models was evaluated using Grad-CAM, Integrated Gradients, and LIME. Localisation quality was assessed by comparing the generated explanation maps with the corresponding tumour segmentation masks using Intersection over Union (IoU), the Dice coefficient, and the Pointing Game.
The average localisation results are summarised in
Table 3. Although both CNN architectures achieved high classification accuracy, the localisation metrics were considerably lower. Among the investigated explainability methods, Grad-CAM produced the highest localisation scores for both CNN models.
DenseNet121 performed slightly better than ResNet50 in the localisation analysis. It achieved an average IoU of 21.49%, a Dice coefficient of 30.21%, and a Pointing Game score of 48.33%. Grad-CAM gave the best localisation results for both models, while Integrated Gradients and LIME showed lower agreement with the annotated tumour masks.
The explanation maps were not always limited to the tumour region. In many cases, activations also appeared around the tumour boundaries or in nearby image regions, even though both CNN models classified the images correctly. This suggests that high classification accuracy does not always correspond to accurate localisation.
Localisation quality also varied between tumour types. The best results were obtained for meningioma images, while glioma and pituitary tumours generally produced more dispersed activation maps.
Representative Grad-CAM visualisations are presented in
Figure 11. The comparison between the generated explanation maps and the corresponding tumour masks shows that the CNN models consistently identified tumour-related regions, although activations outside the annotated tumour area were still present in several examples.
These results show that classification accuracy alone is not sufficient to evaluate explainability. Quantitative localisation metrics provide additional information about how well the explanation maps correspond to the annotated tumour regions.
Among the three explainability methods, Grad-CAM consistently gave the best localisation results for both CNN models in
Table 4. Integrated Gradients and LIME performed less accurately, although the difference depended on the localisation metric being considered.
The localisation results were not the same for all tumour types. Meningioma images consistently produced the best agreement between the explanation maps and the tumour masks. The highest values were obtained with DenseNet121, which reached an IoU of 46.86%, a Dice coefficient of 61.70%, and a Pointing Game score of 90%.
Glioma and pituitary tumour images proved more difficult. In these cases, the activation maps were generally less concentrated around the annotated tumour region, leading to lower localisation scores. Despite these differences, the relative performance of Grad-CAM, Integrated Gradients, and LIME remained similar across all tumour classes.
Representative examples of the generated explanation maps are shown in
Figure 11 and
Figure 12.
Although both CNN models achieved very high classification accuracy, the localisation quality differed considerably between the investigated explainability methods. In other words, a correctly classified image does not always produce an accurate explanation map. For this reason, classification accuracy should be considered together with quantitative localisation metrics.
4.4. Effect of Image Preprocessing
To evaluate the effect of the proposed preprocessing pipeline, both CNN models were retrained after applying Otsu thresholding, binary hole filling, and brightness–contrast jittering.
After preprocessing, Grad-CAM showed better agreement with the annotated tumour regions for both CNN architectures. In contrast, Integrated Gradients produced only minor changes in the IoU and Dice coefficient, while the Pointing Game score slightly decreased.
The average localisation results after preprocessing are presented in
Table 5, whereas the corresponding per-class results are given in
Table 6.
For the DenseNet121 model, the average IoU increased from 21.49% to 23.08%, the Dice coefficient improved from 30.21% to 32.49%, and the Pointing Game score increased from 48.33% to 51.67%. Similar improvements were observed for the ResNet50 architecture. In addition, the DenseNet121 model achieved a validation accuracy of 99.6% after preprocessing.
The preprocessing steps that are used in the proposed framework are shown in
Figure 13, including Otsu threshold masking, binary hole filling, mask application, and RGB conversion before CNN classification and explainability analysis.
The Grad-CAM examples presented in
Figure 14 and
Figure 15 show fewer activations in the background and a better concentration of the highlighted regions around the tumour. This effect is particularly visible for the meningioma and glioma examples.
The preprocessing pipeline shown in
Figure 13 was designed to suppress irrelevant background activations and improve localisation consistency in the subsequent Grad-CAM analysis.
An additional experiment was carried out to evaluate the contribution of each preprocessing step. Three preprocessing configurations were compared: the original MRI images, preprocessing without binary hole filling, and the complete preprocessing pipeline. The average localisation results are presented in
Table 7.
For both CNN architectures, the complete preprocessing pipeline gave the highest Grad-CAM localisation scores. When the binary hole-filling step was omitted, all localisation metrics decreased slightly. This indicates that the hole-filling step helps preserve a more complete brain region before CNN training.
The improvements were moderate but appeared for both CNN models. Preprocessing reduced activations outside the tumour region, although localisation errors were still present. Pituitary tumour images remained the most difficult cases, with activation maps that were generally more dispersed than those observed for glioma and meningioma.
Overall, the results show that conventional preprocessing improves the localisation of Grad-CAM explanation maps, but preprocessing alone is not sufficient to eliminate all localisation errors.
The comparison between the three preprocessing configurations showed that the complete preprocessing pipeline produced the highest Grad-CAM localisation scores for both CNN architectures. Omitting the binary hole-filling step slightly reduced the IoU, Dice coefficient, and Pointing Game values, particularly for ResNet50. Although the improvements were moderate, the same trend was observed for both CNN models.
To determine whether these improvements were statistically significant, the Wilcoxon signed-rank test was applied to the localisation metrics before and after preprocessing. A significance level of
p < 0.05 was used throughout the analysis. The overall results are summarised in
Table 8.
For Integrated Gradients, no statistically significant differences were found. This is consistent with the relatively small changes observed in the localisation metrics after preprocessing. For Grad-CAM, the analysis performed on the complete test set also did not reach statistical significance. This indicates that the overall improvement remains relatively small when all tumour classes are analysed together.
To better understand this behaviour, the statistical analysis was repeated separately for each tumour class. The corresponding results are presented in
Table 9. For ResNet50, statistically significant improvements were found for the IoU and Dice coefficient, whereas the Pointing Game did not reach statistical significance. For DenseNet121, the Pointing Game showed statistically significant improvement, while the IoU and Dice coefficient changed only slightly.
The largest improvements were observed for the glioma class, where all three localisation metrics increased after preprocessing. In contrast, the pituitary tumour class showed only limited improvement. In this class, the IoU and Dice coefficient remained lower after preprocessing, which explains why the statistical significance became weaker when the complete test set was analysed.
Overall, the statistical analysis indicates that the proposed preprocessing pipeline improves the localisation of Grad-CAM explanation maps. Although the improvements are moderate, they are consistent across repeated experiments and become statistically significant for several tumour classes.