Diagnosing Shortcut Learning in CNN-Based Photovoltaic Fault Recognition from RGB Images: A Multi-Method Explainability Audit

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper tackles an important and relevant problem. Unfortunately, both the abstract and introduction are too lengthy. In general, the organisation of the paper requires more clarity.

The abstract should specify what constitutes the major scientific advancement of this paper as compared to previously published work in the field of explainable AI with respect to the recognition of photovoltaics' faults; in other words, state clearly what the novel contributions of this research are.

The reason behind choosing these five architectures should be evident, as well as the rationale for their selection (Baseline CNN, VGG16, ResNet50, InceptionV3, EfficientNetB0).

 

Also, the introduction is too detailed and full of literature for an introductory paragraph. In particular, the numerous sections within Section 1.1—especially those that include an abundance of method-level descriptions—should become less lengthy and condensed. Even though both the research gap and the novelty of this research are present, these elements do not appear until very late in the introduction. This would help the reader follow along with the flow of the paper. To improve clarity and help guide readers through the paper's organisational structure, a more concise writing style can help eliminate repetitions (for example, repeating statements about CNN opacity and the issue of shortcut learning).Please list the novelty and contributions.

In section 2, please cite the dataset.

Please add samples of the images in the dataset.

Section 2 should be named Materials and Methodology.

The structure and organization of the paper need careful revision

A figure should be added to describe the steps of the proposed framework

Abbreviations should be defined only once, when they are first mentioned in the main body of the manuscript,

Performance metrics should be moved to the results section.

Why did you choose only LIME in this study, while there are several XAI methods.

Results should be presented in a separate section. Also, there should be a separate discussion section.

Comparisons with other XAI methods and other CNNs or ViT is needed.

Author Response

Please see the attachment for the author's answers to Reviewer 1 comments

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

This research focuses on the shortcut learning diagnostic problem in RGB image recognition of photovoltaic panel faults based on convolutional neural networks. Five CNN architectures were compared, and an auditing system was built using three complementary and interpretable AI methods. The authors should prepare a major revision to improve the submission. Some comments are listed as follows.

 

1. The article relies heavily on charts to support its conclusions, but the main text only describes the chart trends without specifying key quantitative data.
2. The study explicitly states that data augmentation will not be performed, citing the need to avoid introducing artificial variability and bias, but it does not adequately demonstrate the necessity of this decision.
3. The criteria for selecting the hyperparameters for the three XAI methods are not explained.
4. The study found that cleaning and electrical fault types are susceptible to shortcut learning, but did not delve into the underlying visual feature mechanisms.
5. Data decentralization is an important issue in the field. The authors could discuss this by considering some works. -Balance recovery and collaborative adaptation approach for federated fault diagnosis of inconsistent machine groups.

Author Response

Please see the attachment for the author answers to Reviewer 2 comments.

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

The article investigates shortcut learning in CNN-based photovoltaic (PV) fault recognition from RGB imagery and proposes a multi-method explainability audit combining LIME, occlusion sensitivity, and Integrated Gradients. The study evaluates whether high predictive accuracy reflects physically meaningful evidence or reliance on contextual artifacts.

The characterization of the proposed framework as an “audit” may be misleading, as the study does not define explicit audit standards, thresholds, or compliance outcomes. As presented, the work constitutes a thorough interpretability analysis rather than an audit in the strict sense. Clarifying the operational definition of an “audit” in this context, or revising the terminology accordingly, would improve conceptual accuracy.

The main question addressed is whether CNN-based PV fault recognition models rely on intrinsic, fault-related visual cues or instead exploit spurious contextual correlations (shortcut learning), and how such behaviour can be systematically diagnosed using a quantitative, multi-method explainability framework.

Originality lies on diagnosing shortcut learning rather than improving classification accuracy, and its systematic integration of multiple techniques evaluated using quantitative metrics.

Possible improvements : Results are based on a single stratified 80/20 train–validation split. Repeated splits or cross-validation might improve robustness and findings generalizability. Explicit experiments manipulating background or context might more directly test shortcut learning hypotheses.

In terms of references, recent surveys on saliency faithfulness evaluation may be taken into account.

Minor methodological clarification and additional explanation concerning robustness and clarity, as well as the term "audit" are suggested.

Good continuation !

Author Response

Please see the attachment for the author's reply to Reviewer 3 comments.

Author Response File: Author Response.docx

Reviewer 4 Report

Comments and Suggestions for Authors

Good, serious, analytical, and practical research. The topic of this article is interesting and meaningful. The article examines a currently important and topical problem in the field of computer vision and renewable energy – convolutional neural networks (CNN)-based damage recognition of photovoltaic panels and the challenges associated with it. The results obtained are excellent.

The design of the article is well structured (but it should be noted that the article is difficult to read):

• The introduction part with the literature review is given.
• Descriptions of the datasets, preprocessing, and explainability framework are given.
• The experimental results and their analysis are given in Section 3.
• The conclusions and future work parts are given.

The experimental part is very impressive.

There are no significant criticisms about the research methodology.

 

1. What is the main question addressed by the research?

The main advantage of the article is the systematic and numerically based integration of XAI methods. A multi-method XAI explainability audit is proposed, combining LIME, occlusion sensitivity, and integrated gradients approaches, to analyze the data on which these methods base their decisions.

2. Do you consider the topic original or relevant in the field? Does it

address a specific gap in the field?

The article makes a significant contribution to the application of explainable artificial intelligence in industrial vision and renewable energy systems, which is essential for research in this field.

 

3. What does it add to the subject area compared with other published

material?

 It is valuable to obtain results that show that high classification accuracy (e.g., in the case of ResNet50) does not necessarily imply areas of visual defects. This is a good contribution to research in this area.

 

4. What specific improvements should the authors consider regarding the

methodology? What further controls should be considered?

There are no criticisms about the research methodology. At times, the article becomes very technically dense (e.g., in Section 3), which may be difficult for readers without specialized XAI knowledge to comprehend. A limiting drawback could be considered the small data sample in the experimental part (875 images), which limits the generalizability of the models.

It would be advisable to number the formulas.

 

5. Are the conclusions consistent with the evidence and arguments presented

and do they address the main question posed?

The conclusions are logically based on the results obtained, and it is clearly stated why specific algorithms have been chosen for implementation in practice. The conclusions are consistent with the essence of the article. A direction for future research is included.

6. Are the references appropriate?

References are correct, but not found [14].

 

7. Please include any additional comments on the tables and figures.

Tables and figures are formatted according to the article requirements.

Author Response

Please see the attachment for the author's reply to Reviewer 4 comments.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The introduction section is still lengthy and needs improvement.

Please move Figure 1 to the methodology.

The resolution of Figure 1 is poor and needs enhancement.

please move limitations and future work to the discussion section

 

Author Response

Author's response to Reviewer 1 comments in the attached file

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have well addressed all my comments, and it can be accepted now.

Author Response

Response to Reviewer 2 comments in the attached file.

Author Response File: Author Response.docx

Round 3

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have addressed my comments. Thank you