Review Reports - Unmasking Biases and Reliability Concerns in Convolutional Neural Networks Analysis of Cancer Pathology Images

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The abstract refers to “thirteen highly used cancer benchmark datasets” without clarifying the diversity in modality, resolution, class balance, or patient-level independence, all of which are critical in medical imaging studies.

In section "Introduction", several paragraphs reiterate similar points about CNN opacity, performance-based evaluation, and bias. This repetition reduces narrative efficiency and dilutes the main argument. Reduce repetition and consolidate discussions of interpretability and evaluation into a single, focused subsection. Introduce a taxonomy of bias early in the introduction and explicitly situate the proposed experiment within that framework. Clearly articulate the novel contribution, such as scale (13 datasets), modality diversity, or systematic spatial cropping. Add a short paragraph explaining how such biases could lead to false confidence in clinical AI systems, even during external validation. Reduce redundancy by consolidating background information on CNNs and focusing earlier on the study’s novel angle. Explicitly state how this work differs from existing shortcut-learning and spurious-correlation studies rather than implying novelty through scale alone.

The choice of fixed 20×20 pixel crops is arbitrary and not motivated by prior literature or exploratory analysis. It is unclear whether these patches are resized before input into the CNNs, and if so, how interpolation artifacts may influence results. Additionally, cropping fixed spatial locations across all images risks introducing systematic correlations that CNNs can easily exploit. Explicitly describe preprocessing steps (e.g., resizing, normalization) applied to the cropped images. Consider random or multiple background sampling per image to reduce spatial bias. Validate the “non-informative” assumption using human expert review or texture-based baseline models.

Justify the choice of five epochs or adapt training duration per dataset using early stopping. Discuss the implications of ImageNet pretraining and consider ablation experiments with randomly initialized models. Include variance estimates across multiple training runs to assess robustness.

In several cases, class imbalance is extreme, yet no corrective strategies (e.g., reweighting, stratified sampling) are discussed. Explicitly address class imbalance and report balanced metrics where appropriate. Audit all dataset tables for consistency and correctness. Discuss whether observed background-based accuracy correlates with dataset imbalance rather than true shortcut learning.

Report balanced accuracy, sensitivity, specificity, or AUC alongside raw accuracy. Include statistical tests comparing cropped-image performance to chance-level classifiers. Report variance across multiple training runs to demonstrate robustness.

Expand the limitations section to include:

- the arbitrary choice of patch size and locations,
- reliance on ImageNet pretraining,
- absence of explainability or saliency analyses.
Propose practical mitigation strategies, such as:
- background masking,
- artifact-aware preprocessing,
- cross-dataset validation,
- annotation of acquisition metadata.
Suggest explainability methods (e.g., attribution maps, feature visualization) as tools for identifying bias sources.

Author Response

We would like to thank you for the time they spent reading and commenting on the manuscript, and for the very useful comments. The replies for each of the comments and the corresponding changes made to the manuscript are specified below each comment. The paper is much more detailed now, with some new experiments and new results added based on the comments. For convenience, changes made to the manuscript are highlighted in bold font.

--Author response: We added more information about the selection of the datasets. That has been added to Section 2. We basically selected these datasets since these are the most commonly used datasets in CNN for cancer pathology. Datasets that were cited thousands of times to test whether they are reliable for that purpose. But we also explain their diversity in the tables 1 through 4. That has also been added to the new information added to Section 2. The datasets we chose actually cover the basic modalities used for cancer pathology, including microscopy (with H&E staining), X-rays, CT, ultrasound, and digital cameras as used for dermoscopic imaging.

--Author response: It is absolutely true that the Introduction section had repetitions and redundancies, and at the same time some other information was missing. The Introduction section has been revised substantially. Redundancies and repetitions has now been removed. We also added to the end of the Introduction section a few new paragraphs about the novelty of the study and its impact, and very brief statement about the conclusions. It is not as detailed as the discussion in the end of the paper, but it gives some information and some motivation for the paper. In general, the Introduction section is now much easier to read, and provides more information that was not included in the Introduction of the original submission. We also separated the Introduction section into a "related work" sub-section, so that the reader can get to the innovation of the paper without having to go through the related work, which will only appear later.

--Author response: The 20x20 size was based on previous work using other datasets such as face recognition benchmarks and some biomedical benchmarks that are not cancer. References have been added to the part where the size is introduced. That has also been added to the conclusion section with a short discussion. So the size was picked based on previous results. The images were resized to fit the architecture (which is the common CNN practice), and that has been added to the paper too. Since all images are processed in the same way, artefacts that are driven by the image processing should affect all classes equally, and therefore should not lead to differences between the classes. That note has been added to the conclusions.
As for the texture, while this paper shows that the bias exists, we are now working on understanding *why* it exists. That will include the use of engineered features and explainable AI, but that is a very challenging task that will require a completely new project. We do not yet have the full results, but even if we did we cannot include it in this paper because this paper is already very long. We do describe it briefly in the future work discussion in the last section of this paper.

--Author response: Several things were added. First of all, we added some experiments with 15 epochs. These include experiments with both random weights and transfer learning, and we compare the results. For instance, Figure 21 shows the results when using 15 epochs. We also added experiments that show the accuracy as a function of the number of epochs. For instance, Figure 25 shows how it changes based on the number of epochs, from 1 to 15 epochs. As the graphs show, with transfer learning the accuracy goes up very quickly, in most cases before 5 epochs are completed. But we also show what happens after 15 epochs. So it is correct that 5 epochs was selected somewhat arbitrarily (we had to pick something), but now we have results also for 15 epochs and we show what happens as the number of epochs grow.

--Author response: That is a good point, and it has been done. We added experiments when the classes are balanced. Graphs and results have been added to the paper. A lot more figures have been added to the paper, with more results and discussions. The discussions are added to the paper to the places where the figures are described. For instance, Figures 3, 9, and 15 show results when the classes are balanced. As the figures show, the higher-than-mere-chance accuracy is also present in these experiments, with no substantial difference from the other experiments but it is a good idea to show these results too.

--Author response: Many more results have been added. The precision and recall (specificity/sensitivity) have been added, and results with balanced datasets have also been added. All experiments are compared to mere chance accuracy. The new figures 3, 9, and 15 now show the precision, recall, and F-1. So are Figure 22 and 24, as well as Table 5 all show the precision, recall and f-1 in addition to the classification accuracy.

Expand the limitations section to include:

the arbitrary choice of patch size and locations,

reliance on ImageNet pretraining,

absence of explainability or saliency analyses.

Propose practical mitigation strategies, such as:

background masking,

artifact-aware preprocessing,

cross-dataset validation,

annotation of acquisition metadata.

Suggest explainability methods (e.g., attribution maps, feature visualization) as tools for identifying bias sources.

--Author response: Several things were done. We added to the paper a description of the limitations of the method as well as future work. These include the points made here. Future work also includes using explainable AI. This paper is focused on the observation of the bias and its prevalent, and is already very long and detailed, so adding a new topic in the same paper becomes impractical. But the final section now includes information about future work, and specifically the is of explainable AI.
We also add explanations about the selection of pre-training, which is the common way of using CNNs. But we also added experiments using random weights, and compare the results to transfer learning. It is not surprising that transfer learning provided faster learning.

Reviewer 2 Report

Comments and Suggestions for Authors

In this study, thirteen cancer datasets were analyzed using four common CNN architectures. The authors compared the accuracy of each model with that of the datasets after making cropping segments from the background of the original images that do not contain clinically relevant content. The results show that the CNN models provided high accuracy when using the cropped segments, even though they lacked biomedical information. However, the topic is interesting. Extensive revision is required. Detailed comments are given as:

In the abstract section, the authors should explicitly mention the four cancer types evaluated in the study within the abstract, along with the corresponding CNN performance results. Including these details would improve clarity and allow readers to better understand the scope and effectiveness of the proposed approach.
In the introduction section, please explain the importance and innovation of the proposed solution. In addition, please clearly state the main contribution of the manuscript.
The authors should add a related work section. The current state of the manuscript does not sufficiently cover recent and relevant studies, particularly those addressing bias and reliability in CNN-based cancer pathology analysis.
The authors should provide a more detailed description of the methodology used to crop segments from the background of the original images. Specifically, the criteria, preprocessing steps, and any automated or manual procedures involved should be clearly explained to ensure reproducibility and to assess the potential impact on model performance and bias.
The authors should present the experimental results in a clear tabular format. Organizing the results in tables would improve readability and allow readers to more easily compare performance across models and cancer types.
The authors should include additional evaluation metrics beyond accuracy to comprehensively assess CNN performance. Metrics such as precision, recall, F1-score, and possibly AUC should be reported to provide a more reliable evaluation of the models.
All figures and tables should be improved for clarity and readability. The authors are encouraged to enhance resolution, font size, and formatting to ensure that the presented results are clear and easily interpretable.
The authors should include a Conclusion section and explicitly discuss the study’s limitations. A clear summary of the main findings, along with the limitations of this study, would improve the completeness of the manuscript.
The manuscript would benefit from a thorough language editing to correct grammatical errors and improve readability

Author Response

--Author response: Thank you for reading and commenting on the paper. The comments have been addressed, and a reply to each comment and the corresponding changes made in the manuscript are below the comments. The paper is much longer and much more detailed now. For convenience, the changes in the manuscript are highlighted in bold font. The paper is now much more detailed, and also somewhat longer than the original version.

In the abstract section, the authors should explicitly mention the four cancer types evaluated in the study within the abstract, along with the corresponding CNN performance results. Including these details would improve clarity and allow readers to better understand the scope and effectiveness of the proposed approach.
In the introduction section, please explain the importance and innovation of the proposed solution. In addition, please clearly state the main contribution of the manuscript.

--Author response: The paper shows a very high number of classification accuracy results, for the different combinations of datasets, architectures, and parts of the images. The number of results is far too high to include all of them in the abstract, so we just to added to the abstract the highest classification accuracy observed. We also changed the Introduction section substantially. We added to the introduction more information, especially in the last few paragraphs, that emphasizes the innovation while also stating the problem and its impact on cancer pathology research. We also added to the Introduction a sub-section about the existing literature as described below.

The authors should add a related work section. The current state of the manuscript does not sufficiently cover recent and relevant studies, particularly those addressing bias and reliability in CNN-based cancer pathology analysis.

--Author response: We added a new sub-section dedicated to related work. While there is some previous work on dataset bias, we are not aware of any previous work on CNN bias specific to cancer pathology. We added relevant information and references to previous studies, and the related work sub-section provides a summary of the relevant previous work.

The authors should provide a more detailed description of the methodology used to crop segments from the background of the original images. Specifically, the criteria, preprocessing steps, and any automated or manual procedures involved should be clearly explained to ensure reproducibility and to assess the potential impact on model performance and bias.

--Author response: We added description to Section 2 that provides more details about the cropping. That also includes the specific software/library that was used, and other pieces of information that makes it clearer. The added parts are marked in bold font in Section 2.

The authors should present the experimental results in a clear tabular format. Organizing the results in tables would improve readability and allow readers to more easily compare performance across models and cancer types.

--Author response: We added Table 5 to show some of the results in a tabular format. Because the paper includes a very high number of experiments, with different combinations of architectures, datasets, and parts of the images, the total number of results is very high. A single table that includes all results would take several hundred lines, and will be difficult to read and understand. We added Table 5 to show some of the results, in a manner that the table fits a single page.

The authors should include additional evaluation metrics beyond accuracy to comprehensively assess CNN performance. Metrics such as precision, recall, F1-score, and possibly AUC should be reported to provide a more reliable evaluation of the models.

--Author response: Yes. That has been added. We added the precision, recall and F1 to many of the datasets, including the ISIC dataset (2016 and 2019), DerMNIST, PathMNIST, and BreakHis. That added a lot more figures to the paper making it somewhat longer. The new figures 3, 9, and 15 now show the precision, recall, and F-1. So are Figure 22 and 24, as well as Table 5 all show the precision, recall and f-1 in addition to the classification accuracy.

All figures and tables should be improved for clarity and readability. The authors are encouraged to enhance resolution, font size, and formatting to ensure that the presented results are clear and easily interpretable.
The authors should include a Conclusion section and explicitly discuss the study’s limitations. A clear summary of the main findings, along with the limitations of this study, would improve the completeness of the manuscript.

--Author response: Some of the figures were not in vector graphic, and that has been corrected. All figures now are in vector graphics, and are clear and not expected to pixelate also when Zoomed in. We did not to combine some of the graphs into single figures so that the total number of figures does not become too high (the paper already contains 26 figures). We also added a discussion about the limitation of the study (in bold font) and some information about future work.

The manuscript would benefit from a thorough language editing to correct grammatical errors and improve readability

--Author response: That is correct. We did make many changes to the language, corrected some typos, and the writing of the paper has been improved. We especially re-wrote and re-organized the Introduction section, but many other changes were made to the other sections.

Reviewer 3 Report

Comments and Suggestions for Authors

Comments:

The authors presented in lines 70-78 (sect.3) several phrases where they explained how they performed four CNN models in predictions of cancer pathology using images from original datasets and created dropped datasets. They did not explicitly present the principal contributions of this research. This reviewer recommends describing the principal contributions obtained by authors in the form of short highlighting phrases focusing on original development of designed approach that can permit a potential reader better understanding novelty of current study.
In sect. 2, the authors wrote “The experiments are focused on automatic classification of radiographs using four commonly used CNN models: ResNet50 [31], DenseNet121 [32], Inception V3 [33], and VGG16 [34].” The models mentioned are limited according to references only their investigations into breath or skin cancers. Please explain for a clarity of a potential reader such selection of investigated architectures as well as chosen datasets.
The authors wrote “These cropped images are 20×20 pixel sub-images from five different parts of each original image: upper-left, upper-right, center, bottom-left, and bottom-right. That is, from each dataset, we create five new datasets with the same number of classes and images. Instead of the original images, the dataset contains just small images separated from the non-medical parts of the original images.” The text presented in lines: 72-78, 130-133 did not give sufficient information about training-validation-testing process in created datasets. The authors should explicitly explain how they used these dropped datasets during training-validation-testing process. Also, they should provide additional information about the same process when they perform classification process via original datasets that have medical information. Maybe it would be useful presenting training-validation-test curves per epoch for loss function or chosen performance measures (f. e., ACC, PRE, F1-score, etc.) that can justify the quality of the training-validation stage as well as testing one. This maybe can explain such biased classification results obtained in cropped datasets.
This expert noticed that the authors when they discussed performance results only Accuracy (ACC) criterion that did not characterize performance of a framework completely. It is known that in classification problems other criteria are also used: Sensibility, Precision, F1-score, Index Balanced Accuracy (IBA). Numerous studies highlight that ACC alone does not guarantee better quality of a model, and additionally it should be necessary the justification of performance, employing at least measure F1-score and IBA that limits imbalance of classes.
Please provide information about experimental settings (hardware, software) used in this study.
The authors should analyze the limitations of their study, and discuss how possible drawbacks can be resolved.

Author Response

We would like to thank you for the time they spent reading and commenting on the manuscript, and for the very useful comments. The replies for each of the comments and the corresponding changes made to the manuscript are specified below each comment. The paper is now longer and much more detailed. For convenience, changes made to the manuscript are highlighted in bold font.

The authors presented in lines 70-78 (sect.3) several phrases where they explained how they performed four CNN models in predictions of cancer pathology using images from original datasets and created dropped datasets. They did not explicitly present the principal contributions of this research. This reviewer recommends describing the principal contributions obtained by authors in the form of short highlighting phrases focusing on original development of designed approach that can permit a potential reader better understanding novelty of current study.

--Author response: Thank you for the comment. This paper is somewhat unusual in the sense that it does not propose a new method, but instead profiles the existing common practices to show that they can lead to bias. Its importance is that it shows that the existing practices that were borrowed from the machine learning world may not be ideal for cancer pathology, and can in practice lead to biased/overoptimistic results. That has been added to the revised Introduction section (Section 1). The entire last part of Section 1 is now dedicated to explaining the purpose and meaning of the paper, which indeed needs explaining given that the paper is not necessarily a "typical" methodology paper.

In sect. 2, the authors wrote “The experiments are focused on automatic classification of radiographs using four commonly used CNN models: ResNet50 [31], DenseNet121 [32], Inception V3 [33], and VGG16 [34].” The models mentioned are limited according to references only their investigations into breath or skin cancers. Please explain for a clarity of a potential reader such selection of investigated architectures as well as chosen datasets.

--Author response: The architectures were selected given that these are the most commonly used CNN architectures, and have also be used extensively in the field of cancer pathology. That has now been added to the paper (Section 2), where the architectures are first introduces, with some references to previous studies using these architectures for cancer pathology. The same is also with the datasets, that were chosen by their highly common use in CNN for cancer pathology. These are the mostly used datasets and the field. Testing their soundness is therefore much more impactful than testing the soundness of a dataset that has barely been used for that purpose. That has also been added to the paper (where the datasets are first introduced).

The authors wrote “These cropped images are 20×20 pixel sub-images from five different parts of each original image: upper-left, upper-right, center, bottom-left, and bottom-right. That is, from each dataset, we create five new datasets with the same number of classes and images. Instead of the original images, the dataset contains just small images separated from the non-medical parts of the original images.” The text presented in lines: 72-78, 130-133 did not give sufficient information about training-validation-testing process in created datasets. The authors should explicitly explain how they used these dropped datasets during training-validation-testing process. Also, they should provide additional information about the same process when they perform classification process via original datasets that have medical information. Maybe it would be useful presenting training-validation-test curves per epoch for loss function or chosen performance measures (f. e., ACC, PRE, F1-score, etc.) that can justify the quality of the training-validation stage as well as testing one. This maybe can explain such biased classification results obtained in cropped datasets.

--Author response: More information has been added to Section 2 to better describe the cropping and testing of the dataset. That includes software used, epochs, optimzer, image resolution, etc. For consistency, the original images are treated in the exact same way each of the cropped image datasets are used, and that has also been added to the section. We added the precision, recall and F1 to many of the datasets, including the ISIC dataset (2016 and 2019), DerMNIST, PathMNIST, and BreakHis. That added a lot more figures to the paper making it somewhat longer. The new figures 3, 9, and 15 now show the precision, recall, and F-1. So are Figure 22 and 24, as well as Table 5 all show the precision, recall and f-1 in addition to the classification accuracy.
We also tested the impact of the number of epochs. For instance, Figure 21 shows the results when using 15 epochs. We also added experiments that show the accuracy as a function of the number of epochs. For instance, Figure 25 shows how it changes based on the number of epochs, from 1 to 15 epochs.

This expert noticed that the authors when they discussed performance results only Accuracy (ACC) criterion that did not characterize performance of a framework completely. It is known that in classification problems other criteria are also used: Sensibility, Precision, F1-score, Index Balanced Accuracy (IBA). Numerous studies highlight that ACC alone does not guarantee better quality of a model, and additionally it should be necessary the justification of performance, employing at least measure F1-score and IBA that limits imbalance of classes.

--Author response: Thank you for the comment. As also mentioned above, we added the F1 , precision, and recall to many of the experiments. That was done for the ISIS dataset with and without transfer learning, and for other datasets too. Several figures and tables have been added as specified above. That makes the paper a bit long, but the information is now included.

--Author response: A lot of information has been added. The cropping of the images was done by PIL, and that has been added to the paper. As for the hardware, we used a fairly powerful machine called Beocat. A reference to a paper describing Beocat has been added to the paper, along with a brief description. Beocat has over 11,000 CPU cores and 170 GPUs. It is a very powerful computing facility, as needed to run so many CNN experiments. That has been added to the revised version of the paper (Sub-section 2.2).

The authors should analyze the limitations of their study, and discuss how possible drawbacks can be resolved.

--Author response: We added to the revised version information about the limitations of the work to the last section of the paper. The main limitation is that it is an empirical study that shows bias, but does not provide full explanation to *why* that bias exists. Since CNNs are known to be complex, and driven by unintuitive rules, it will not be easy to solve that limitation. But we have plans of approaching it, and we added a paragraph about future work that shows an outline of how we plan to approach that limitation.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

First of all, I would like to thank the authors for their efforts in addressing the peer reviewers’ comments. Although the overall quality has improved, many comments were left unanswered or not properly answered.

For example, a comment regarding adding a separate section for the conclusion should be included, with a discussion of the study's limitations and future work.

Author Response

For example, a comment regarding adding a separate section for the conclusion should be included, with a discussion of the study's limitations and future work.

--Author response: Thank you once again for reviewing the paper. We added information about the limitations, future work etc, but it was not part of a "Conclusion" section. We now change it so that there is a Conclusion section that discusses all of that. Paragraph 5 and 6 discuss the limitations of the research. The following paragraphs discusses future work. These are now in a "Conclusion" section.
Other changes to the paper include improving the quality of the figures, including changing the font size. Some of the figures have several panels to reduce to total number of figures in the paper which is already high, but the font sizes are larger and all figures are vector graphics so the resolution is essentially infinite. we also added more tables to show the precision, recall, and F1 of the results in addition to the accuracy.

Reviewer 3 Report

Comments and Suggestions for Authors

In opinion of this reviewer, the authors tried to respond to all comments of this reviewer. They presented numerous additional figures, and the novel version has now 26 figures (previous version had 18 figures). This reviewer thinks that authors should use all presented material, in particular figs. 3, 9, 15, and table 5 where it can be explicitly seen, when they use to characterize quality of classification several metrics (ACC, RECALL, PRE, and F-1 SCORE), the original images give significantly better performance. The authors should discuss these novel quality results with much attention.

Author Response

--Author response: Thank you once again for reviewing the paper. We added five tables like Table 5 (in the previous manuscript), which shows the accuracy, precision, recall, and F1. We prefer to add them as tables because adding figures would require 5x4 figures, which means 20 more figures to a paper that already has 26 figures in it. We also added discussions for each table. It is expected that the original image would give higher accuracy because there is more information there compared to the small background patches, but the background patches that have no medical information also allow to detect cancer (with accuracy higher than random), and that is the unexpected thing that should be taken into serious considerations. It shows that these images contain not just cancer information, but also other pieces of information that boost the classification accuracy without having anything to do with cancer, and therefore the common machine learning practices might show biased results rather than the sole ability of the method to detect cancer. That has been added to the conclusion section. We would note that in some of the experiments the background accuracy provided similar performance to the accuracy when using the original images. But in any case, no-medical parts of the image are supposed to provide random accuracy when attempting to classify cancer, and that is not the case empirically, meaning that the datasets are biased and should be treated with caution.