Multi-Region and Multi-Band Electroencephalogram Emotion Recognition Based on Self-Attention and Capsule Network

Round 1

Reviewer 1 Report

The paper presents a capsule neural network for decoding EEG signals in an emotion task.

The paper is interesting but I have many concerns that should be addressed.

1. I am a bit worried about three concomitant things, the bandpass filter between 4-45Hz the theta band filter between 4-7 Hz and the window of 1s since we are near Nyquist frequency. Given that a minimum of 5-7 cycles is needed in EEG/MEG analyses for effective characterization of signal (Basti et al. 2021).

2. Please, describe more in detail the characteristics of your dataset, how many trials/samples/features. It is not clear to me.

3. Line 118: Please specify what are 40 and 60 in the text.

4. Line 197: How is the cross-validation performed? To claim that your method is cross-subject you must use leave-one-subect-out cross-validation, also to be sure that independence of samples is assured (Varouquaux et al. 2016). Please use this approach or remove references to cross-subject decoding.

5. I am always worried about overfitting of accuracy with neural networks. How many parameters does your network have? Do you have enough samples with respect to parameters?

6. What were the characteristics of the SVM you used? Was it non-linear? I would suggest using non-linear SVM.

7. What is the rationale of using 6 layers of Multi-head Attention?

8. I am always skeptical regarding the fact that we use a parameter of the raw EEG signal thus reducing the power of NNs to extract meaningful features by themselves. Can you comment on this.

9. There are several other papers that used multifrequency information to classify M/EEG signals also with simple classifiers, you should include them in the discussion (e.g. Dimitrakopoulos et al. 2017, Wang H, Wu X and Yao L 2020, Syrjala et al. 2021)

English is ok. But I would improve clarity of exposition.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Dears Authors

General comments

I think this is an article with significant content and of interest. In my opinion, it is an original study and of great interest to readers. Congratulations.

The article has some weaknesses which need to be reflected upon and possibly changed.

However, to make the article even better, I present my reflections according to your article.

Title

“Multi-region and multi-band EEG emotion recognition based on self-attention and capsule network” - Clear and directive

 Abstract

- Clear and directive.

- The sample remains to be described.

- Keywords: appropriate.

 Introduction

- From my perspective, it would be interesting and important, in my view, to theoretically frame/explore the concepts of emotion recognition and self-attention.

- Lines 64-66- “This process not only enhances the capture of potential relationships between local and global features but also enables the classification of emotional states through emotion capsules..”- it would be important to explain this information further.

 - Lines 67-78- You mention that “The main contributions of this paper can be summarized as follows: 1. This paper proposes a new Capsule-Transformer method for emotion recognition of EEG signals, and explores the role and importance of different brain regions and frequency bands of EEG features for emotion recognition. 2. The innovative combination of brain region self-attention, frequency band self-attention, and dynamic routing mechanism for EEG emotion recognition enables the model to not only capture the difference information of emotion features in different brain regions and frequency bands simultaneously, but also further extract the intrinsic relationship between signals from different brain regions in the emotional activities. 3. Based on the proposed method, we conducted subject-dependent experiments on the DEAP public dataset and achieved excellent performance in classifying emotional states.“- I think this information would be clearer if presented in another way. For example: “The innovative combination of brain region self-attention, frequency band self-attention, and dynamic routing mechanism for EEG emotion recognition enables the proposal model to not only capture the difference information of emotion features in different brain regions and frequency bands simultaneously, but also further extract the intrinsic relationship between signals from different brain regions in the emotional activities. This paper proposes a new Capsule-Transformer method for emotion recognition of EEG signals, and explores the role and importance of different brain regions and frequency bands of EEG features for emotion recognition. Based on the proposed method, we conducted subject-dependent experiments on the DEAP public dataset.”

 - Lines 79-81- “The rest of the paper is organized as follows, Section 2 describes the proposed Capsule-Transformer method in detail, Section 3 shows the experimental setup, experimental content, experimental results, etc., and Sections 4 and 5 discuss and conclude the work of this paper, respectively.”- I don't think it's relevant to be presented in an article.

 2. Materials and Methods

- I think that before describing the EEG Dataset, the sample should be presented and described. Just saying "32 subjects (16 males, and 16 females)" doesn't seem enough. And what were the inclusion and exclusion criteria? Considering the type of study they carried out, ensuring exclusion criteria, such as not having a psychiatric diagnosis, not taking psychotropic drugs, not having addictive behaviors, among others, are extremely significant. I believe you must have taken these criteria into consideration, but they are not identified/explained in the paper.

- The investigation procedure should also be presented in detail. There is no information on this.

- Lines 90-91- “valence, arousal, dominance, and liking”- What do you mean? In my opinion, you should frame/support with reference. You can't understand that.

 4. Discussion

-Lines 318-320- “At present, the stability of the Capsule-Transformer method in cross-subject EEG emotion recognition still needs to be improved.”- I think it should be placed in a separate section as limitations of the study. There are other limitations that have not been identified in the text and, in my opinion, it would be essential to list them.

-Lines 320-323- “In the future, we will consider more complex emotion recognition scenarios and collect more emotional EEG data based on this method to further explore the possibility of solving the current deficiencies.”- I think it should be put in a separate section as a suggestion for future studies. And list other suggestions.

- At this point, I think it would also be relevant to briefly explain the importance of your results in clinical practice.

 5. Conclusions

-Lines 329-331- I think the conclusion should be more explanatory. You mention that “the experimental results suggest that more attention should be paid to the frontal lobe region and high-frequency EEG signals in emotion EEG research.” I think objectivity is important here, but your results are not really presented. In my opinion, you should detail the main results of your work.

Thank you for the opportunity to read and comment on your study.

I hope I have contributed to a reflection that can further improve your work.

I wish you good work.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

I am not completely satisfied of the responses regarding question 1 and question 4. 

In question 1, the authors did not address the issue. The problem is that with a short window you are introducing an artifact specially in lower frequencies, but you think that it is better to have more near-artifact features than artifact-free features.

In question 4, they claim to have a cross-subject classifier, but from the response they used a within-subject approach and thus cross-subject classifier. 

Author Response

Please see the attachment.

Author Response File: Author Response.docx