Uncertainty-Aware Deep Learning for Robust and Interpretable MI EEG Using Channel Dropout and LayerCAM Integration

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper considers the important and novel uncertainty-aware deep learning framework for MI-EEG classification. This aims at simultaneously enhancing the model’s robustness and interpretability. To improve the performance,  it involves the channel dropout regularization, layerCAM integration, evaluation across varying montages, and subject grouping. The overall pipeline is valuable. However, there are several points the authors could consider to further improve this paper:

1. Why the introduction of uncertainty module improves the performances, could the authors provide more empirical or theoretical evidence on the working mechanisms of the uncertainty module?
2. According to Figure 2, there are seem to exist an optimal setting for the dropout rate. How could the dropout rate be selected in real practice. If the doctor wants a safe prediction, should the doctor choose larger or smaller dropout rates?
3. Following the above question, if a test set is used for evaluating the optimal dropout rate, then how to ensure there is no out-of-distribution generalization issue, where the real world test dataset possess different distributions as the training data distribution. The author could cite and refer to ood-control tpami, ood-bench cvpr paper to discuss about this.
4. Has the model been deployed in real hospital scenarios? What is the biggest challenge for an EEG model to be really useful in practice and help saving people’s lives.
5. Could the authors provide opensource codes after the acceptance of this paper.

Overall, this is an interesting paper and worth reading. The authors could consider modify this paper for further improvements.

 

Author Response

See attached pdf.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Review Report for The Manuscript applsci-3712729

Dear Editors, 

I have reviewed the manuscript titled “Uncertainty-Aware Deep Learning for Robust and Interpretable MI EEG using Channel Dropout and LayerCAM Integration” The paper addresses a well-known research topic with an intriguing idea. The manuscript is well-written and clearly presented in most of its parts. It introduces a promising framework combining channel dropout with LayerCAM-based interpretability for robust classification of motor imagery (MI) EEG. The authors suggest that the approach enhances both robustness and interpretability of EEG-based brain–computer interface systems and achieves improved classification accuracy compared to SOTA models. The authors also claim that their model demonstrates a fair degree of robustness and a good degree of generalizability. The paper’s content is promising and potentially worthy of publication. However, I have some concerns that I would like to see addressed, and I think some work is still needed before the manuscript is ready for final publication, and I trust that the author will take this step to strengthen the quality of the work. I detail it in the comments below.

Detailed Comments

Methodological Limitations

At the current stage, several issues limit the interpretative strength of the conclusions. The model shows some performance benefits, making it a promising contribution to EEG-based BCI systems. However, since the authors aim to propose their framework as a consistent improvement in EEG-based BCI generalization, they should strengthen the statistical analysis by including formal significance testing (e.g., p-values, confidence intervals, and post-hoc analyses corrected for multiple comparisons) to support performance claims that are currently not reported. Additionally, the rationale behind the hyperparameter choices should be elaborated more to enhance reproducibility. Finally, interpretive claims—such as strong generalization ability—should be tempered or more rigorously supported, particularly in the absence of cross-session or task-transfer validation. Addressing these points would improve the methodological part and validity of the conclusions.

State of the art and References

The Reference list should be substantially improved: a work addressing a broad and extensively studied domain such as motor decoding with deep learning methods should include more than the current total of 48 citations, which are very few. The reference list should integrate prior efforts (some of them pivotal https://doi.org/10.1038/nature04968  ; https://doi.org/10.1038/416141a ; https://doi.org/10.1152/jn.01245.2003 ;https://doi.org/10.1523/ENEURO.0506-19.2020   ) that have been overlooked. In addition to these foundational works, other recent efforts have utilized machine learning techniques, deep learning architectures (especially Transformers), and various computational and analytical frameworks to enhance the decoding and spatiotemporal modeling of neural activity, aiming to predict motor intentions and improve related BCI applications. This recommendation applies to both non-invasive modalities like EEG and fMRI, as well as invasive recordings like intracortical and ECoG. I would appreciate it if the author could consider a few adjustments in this regard, for example, in the “Introduction” or “Conclusion” sections. Here are some: https://doi.org/10.1371/journal.pbio.0000042 ; https://doi.org/10.1088/1741-2552/ac8fb5 ;  https://doi.org/10.1088/1741-2552/acd1b6 ; https://doi.org/10.3390/s25051293 ; https://doi.org/10.48550/arXiv.2206.04727 ; https://doi.org/10.1088/1741-2552/adaef0  ; https://doi.org/10.1016/j.neunet.2009.05.005 ; https://doi.org/10.1038/s41593-019-0488-y ; https://doi.org/10.1515/revneuro.2010.21.6.451 ; https://doi.org/10.1016/j.bspc.2021.103241 . This will provide a more comprehensive contextual and theoretical foundation and prevent the omission of key contributions.

Comments for author File: Comments.pdf

Author Response

See attached file.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Introduction

The introduction gives a broader context to MI-EEG classification problems emphasizing the importance of reliable and interpretable models. The contribution of combining Monte Carlo dropout for uncertainty and LayerCAM for interpretability is made clear.

• The motivation can further be concretized by explicitly citing the research gap: that robustness or interpretability has been tackled in the past, but not both at a combined level.
• The study's contributions are not listed. Finally, finish the introduction with a bullet-pointed list of the study's specific contributions.
• Enhance the transition from the broad MI-EEG problems to the proposed technical advances, with short intuitive descriptions of "uncertainty estimation" and "LayerCAM" for the less experienced reader.

Literature Review

• The manuscript references the pertinent literature on deep learning for EEG, interpretability methods (CAM/GradCAM), and uncertainty modeling. The discussion, however, is blended with other sections and is not provided a separate or thematically organized review.
• The literature review is not thorough and well-organized. Important subjects like explainable AI for EEG, Bayesian deep learning for neuroimaging, and sparse EEG channel configurations need to be deliberated more extensively.
• There is no "Related Work" section. I suggest organizing one with thematic sub-sections:
  • Deep Learning in MI-EEG
  • Uncertainty Estimation in Neural Models
  • Explainable AI in EEG
  • Sparse Electrode Configurations
• Explain how this work builds on existing work by combining both interpretability and uncertainty in a single pipeline.

Methodology
The approach entails training various deep learning models on the BCI Competition IV dataset with varying channel densities (8, 16, 32, 64). Channel-wise Monte Carlo Dropout (MCD) uncertainty quantification and LayerCAM spatial attention interpretation are employed. Model and montage performance comparison are done.

• There is no clear description of training protocols in the methodology, e.g.:
• Cross-validation strategy (within- or between-subject)
• Optimizer, learning rate, batch size, number of epochs
• Regularization techniques and hyperparameter search
• No ablation study is presented. It is hard to disentangle the contribution of channel dropout and LayerCAM separately.
• A diagram of the model would clarify where MCD and LayerCAM are being inserted.
• Include more information about how LayerCAM is calculated (e.g., which layer, which normalization, etc.).
• Specify whether interpretability results were evaluated per trial, per subject, or pooled.whether interpretability outputs were analyzed per trial, per subject, or aggregated.

 

The article displays mean classification accuracies and performance improvement per model and per channel configuration. It does not apply formal statistical testing or error estimation techniques.

There is no application of statistical significance testing:

• Use Wilcoxon signed-rank tests for comparing models in pairs. Use Friedman test for comparing more than two conditions.
• No confidence intervals are reported. Use bootstrapping or MCD-based resampling instead.
• No inter-subject variance analysis is provided. Provide subject-specific boxplots or variance tables.
• Effect sizes (e.g., Cohen's d) must be reported in order to assess the size of improvement.
• Uncertainty scores based on MCD are not statistically compared or contrasted with model performance, although they represent the main contribution.

 

• Provide reliability diagrams, predictive entropy, or Brier scores.
• The interpretation only includes descriptive summaries with no further comprehension of patterns or failure cases.
• LayerCAM visualizations are displayed but not explored systematically or verified.

 

• Include comparisons of saliency maps across subjects/models.
• Verify if saliency corresponds to motor cortex areas.
• Interpretability is given qualitatively. Offer quantitative measures (e.g., overlap with ground truth areas or typical saliency evaluation).

 

• Results need to be better structured regarding research questions or hypothesis-driven statements (e.g., "How does model uncertainty correlate with prediction error?").

 

Discussion
Discussion is combined with result/conclusion sections and brief. Restates results but is not comprehensive.

• Discussion needs to be lengthened to include:
• Comparison with current literature
• Theoretical implications of interpretability and shared uncertainty
• Limitations (i.e., single dataset, lack of external validation)
• Future research directions such as cross-subject generalization, online deployment, or multi-modal EEG fusion

Author Response

See attached pdf.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have addressed all of my major comments, and I appreciate their thoughtful revisions and clarifications. However, there are still some inaccuracies that should be corrected to ensure the manuscript meets the standards. In particular, the authors should carefully check all references to verify that bibliographical information is complete and accurate; several entries are missing volume or page numbers, report incorrect years, or lack DOIs (e.g., Candelori et al. and Arif et al. both miss volume and page numbers; Candelori et -al. is from 2025 and not 2024. And many others).  This should be punctually verified. Additionally, I noticed a few typographical and spelling errors throughout the manuscript that should be corrected in the final version.

Author Response

See the attached pdf.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The authors have more than adequately addressed all my comments and concerns.

Author Response

See the attached pdf.

Author Response File: Author Response.pdf