Review Reports

Round 1

Reviewer 1 Report

AUTHORS COMMENTS:  

  

The article presents a method for automatic recognition between Parkinson’s disease (PD) subjects and healthy subjects (young and eldery) based on gait analysis using inertial sensors and deep learning approaches, including a muti-input sequential and CNN based method. This study is not very novel and most of the results are not particularly surprising (similar performances among models). However, is a useful contribution for PD monitoring using inertial sensors. In addition, some ideas discussed should be supported by the related literature, and some information about the prediction models is missing to facilitate the reproduction of the results. The structure of the article can be improved to facilitate the reading, and few minor typo-errors should be addressed. The paper is suitable for publication if authors could address the following points:

 

  

SPECIFIC COMMENTS:  

  

Point 1: The introduction does not discuss the applicability of gait analysis, instead it only mentions "automatic and continuous assessment of motor symptoms of PD". A brief discussion explaining gait analysis using inertial sensors should be included, since gait is one of the most important indicators of motor competence in PD.  

  

Point 2: A brief description of the importance of wearables and inertial sensors in PD/gait is missing, as well as a description of how they differ from other methods based on sensors (wearable & non-wearable).  

  

Point 3: In the contributions section, some results are reported (i.e., "According to our results, GRUs were better than CNNs regardless of gait task"). Instead, it is suggested to highlight the novel methods proposed in this study.  

  

Point 4: It is not explained the reason for the selection of the sample rate and bit depth of 102.4 and 12 bit respectively, considering that the gait analysis with sensors placed in the lower extremities is focused on low frequencies and therefore can be performed with low sampling rates.  

  

Point 5: In section 2.3 the description of the CNN lacks a reference as presented in the GRU section. The same applies to the description of the pooling layer (lines 163-166).  

  

Point 6: The results section presents part of the methodology related to the implementation of the deep models. It is suggested that these sections be properly ordered to facilitate reading. Or improve the description of the headings and subheadings of this section.  

  

Point 7: It is not clear the parameter optimization process, nor the parameters that were tuned in the different models.  

  

Point 8: The authors report that they did not apply pre-processing. Did the authors consider appling some kind of normalization to the input data (exemplar 0-1, Z-score normalization), being a process commonly applied in deep-learning-based approaches? 

 

   

Point 9: The description of the hyperparameters used in the predictive models (number of units, kernel size, # filters, etc) is missing. The lack of this information makes it difficult the reproduction of the results.   

   

Point 10: In the discussion section, it is mentioned "Previous work suggests that CNN architectures are a suitable approach when considering STFT of gait signals" however, no reference is included to support this statement.  

   

Point 11: A brief discussion of methods and possible solutions that can be employed in future studies to address the lack of data (i.e., Transferlearning, data augmentation) can be included in the discussion.  

  

Minor issues:  

  

In line 169: Stochastic gradient descent (SGF) -should be SGD  

  

In section “2.4. Gate Recurrent Network, GRU”, following text 

“… Among the advantages of the GRU over other networks…” should be changed to “...Among the advantages of the GRU over other recurrent networks”. 

 

The formulas used to calculate the performance metrics are not presented, or at least these equations should be referenced.

 

The shape of the tensors used to feed the models is not described.  

 

In Fig 12, The line colors on AUC curves are difficult to distinguish

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

In this paper, a classification method of Parkinson's disease patients based on deep learning strategy is proposed, and motion information are collected through IMU sensors installed on the heels of shoes. In the experiment, the classification effects of three kinds of deep learning algorithms are compared under two scenarios (PD vs EHC and PD vs YHC). This study is helpful for clinical diagnosis and has certain medical value.

The following points need to be addressed:

1.     The classification effect of the combination of CNN and GRU model seems to be almost the same as that of GRU model. The author interprets it as that the amount of data is small, which seems to be unconvincing; It is unreasonable to propose a new method without fully verifying its effectiveness.

2.     In section 2.2, only the acceleration and rotation speed data at the heel are collected, and the motion information of the leg is ignored, which is not comprehensive, and may also affect the classification effect of deep learning model.

3.     In section 2.3 and 2.4, only CNN and GRU models are introduced. However, CNN and GRU are used to process image data and sequence data respectively, and the fusion logic of these two models is not clearly expressed.

4.     Please supplement the hyper-parameters of CNN and GRU model.

5.     It is suggested to present the results of CNN, GRU and fusion methods into one table, which may be more intuitive.

6.     The model output is missing in Fig 1. The CNN architecture in Fig 5 and Fig 9 is inconsistent；

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper at hand presents classification of results based on three machine learning approaches to identify Parkinson’s disease in two different walking scenarios. Subjects have been split into three different groups: One group with subjects suffering from PD and two healthy groups, split by age.

The authors collected a fair amount of data with samples from more than 130 subjects. Thus, findings reported in this work should be reproducible. The overall structure of the paper is clear and well done.

However, I have two points where the paper needs to be improved before it is ready for publication:

First, in some places the work would benefit from some restructuring and addition of details: For instance are the walking tasks already talked about (line 103/104) before they are described in a later section. The last column in Table 1 is not explained. The paper mentions the segmentation of the motion data into snippets before a description is provided. The description of the segmentation is vague and could benefit from more details: Overlapping window size. Also, why are no motion segmentation techniques applied instead of cutting the data in uniform samples?
Figure 7 shows Gyroscope data, but no Acceleration data are shown in the paper. Abbreviations are used without definition (STFT in line 230).
I disagree with the authors, that the presented approach doesn’t use a feature extraction technique for two reasons: a) Even if raw data is used, they are used as features.  b) the computation of a spectrum is a complex feature extraction from the time series data.
The authors don’t mention why they choose a unusual sampling rate of 102.4Hz for their data collection.

Second, I disagree with the argumentation of why the combination of the two presented techniques doesn’t obtain better results. The authors claim this to be based on the small size of the dataset. However, as stated above, I think the dataset has a reasonable size. I assume that both approaches, although they are quite different, come to the same criteria to separate the motion types in question. Maybe the CNN approach can’t split as good as the GRU approach, but if they found the same splitting criteria, they can’t improve the results. This would be different if one would compare classification betweenaccelerometer based, gyroscope based and combined features, as the different sensor modalities capture different aspects of the motion. Since the statement of the small data being responsible for not improving the results is quite prominent in the paper, I would ask the authors to reconsider and adapt the paper if they can’t find arguments supporting this argument.

That said, I think the paper is not yet ready for publication, but can be significantly improved in a major revision.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

REVIEWER COMMENTS: 

 

I am glad the authors followed my suggestions in almost all the parts. I also understand that some of my requests were hard to fulfill in the present document. In summary, the structure and the writing of the article are adequate and can be considered for publication after reviewing the following minor errors: 

 

Specific comments: 

 

Point 1: Minor typo error in title 1.1  

“…1.1. Sate of the art...” 

 

Point 2: 

The title of Figure 1 is not clear. It is suggested to extend and clarify the description. 

 

Point 3 

The statement in lines 131 to 134 is not correct. “The sensor used, by default, allows capturing motion patterns at a sampling frequency (Fs) of 102.4 Hz with 12 bits of resolution. The reason for this Fs value is because it allows computations up to powers of 2”. 

 

This statement should be rectified. If I understand what the authors are referring to, the selection of values with powers of 2 is related to the size of the window (to allow the application of the FFT instead of the DFT), rather than a strategic value to select the sampling frequency (which is not the case for an Fs of 102.4 Hz). Instead, it should be clarified that the selection of this Fs (and bit depth) is a consequence of the acquisition device, and it could be discussed that the sampling frequency could be lower to allow the implementation of optimized predictive models with a reduced computational burden. 

  

Point 4:  

In line 271 the definition of “Adam optimizer” needs a reference. 

 

Point 5: 

A minor typo error has been found in Figure 12: “...Figure 12. Comparison of the best results considerin the 4 × 10 m task and both feet.”

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

All questions have been answered. We hope that the method proposed in this paper can be applied in the diagnosis of Parkinson's disease.

Author Response

Thank you very much for your comments.

Reviewer 3 Report

The authors have integrated changes to the manuscript for most of my comments. Thus, the paper has been clearly improved. 

The separation of accelerometers and gyros was just meant as an example, not as a task for an iteration. Therefor, I didn't want the authors to include this analysis to the paper. 

Author Response

Thank you very much for your comments.