A Low-Cost Video-Based System for Neurodegenerative Disease Detection by Mobility Test Analysis
Round 1
Reviewer 1 Report
The paper seems good but I am not sure about the content. I think there are some points in the manuscript that need more explanation and better presentation.
My comments and suggestions are the following:
- The superiority of your model should be analyzed in a better way.
- The experimental setup should be presented in a better and more comprehensive way.
- You should separate the discussion from the conclusions. In addition, the discussion should be analyzed in more detail.
- You should include more up-to-date references.
- Please explain more about your future plans.
Author Response
The authors would like to thank the reviewer for his/her very constructive and informative suggestions and comments. In the following, we provide a point-by-point response. In the manuscript, all the principal revisions are highlighted in red.
-The superiority of your model should be analyzed in a better way.
The introduction has been revised to clarify the main novelty of the paper.
In particular, we do not propose a new model but a whole framework that can
be used to solve a real problem emulating the analysis that physicians perform
while older people execute the StS Test. The whole framework consists of a low-cost vision-based set-up and modules for the acquisition, recording, processing, and analysis of videos. Then, we define different features and compare different machine learning methods. These comparisons brought us to the final proposal of the best combination of features and the Machine Learning method, which provides the best results in this real context.
- The experimental setup should be presented in a better and more comprehensive way.
The experimental setup has been better detailed in section 2. Furthermore,
Figure 2 has been added to give a schematic view of the system.
- You should separate the discussion from the conclusions. In addition, the
discussion should be analyzed in more detail.
Discussion and conclusive remarks have been reported in two separate sections as suggested.
- You should include more up-to-date references.
More up-to-date references have been included and the introduction section
has been opportunely modified.
- Please explain more about your future plans.
The conclusion section has been completely revised including our future
plans.
Reviewer 2 Report
The article is devoted to developing an automatic visual system based on an inexpensive camera that can be used to support medical personnel in diagnosing neurodegenerative diseases and support mobility assessment processes in the context of telemedicine. The study's relevance is justified by the fact that monitoring mobility tests can significantly help diagnose neurodegenerative diseases. In particular, among the various mobility protocols, the Sit-Stand (StS) test was very significant since its performance, both in terms of duration and postural assessment, can indicate the presence of neurodegenerative diseases and their degree of progression. At the same time, the assessment of the StS test mainly depends on the experience and qualifications of the medical staff. Therefore, in this article, the authors propose an inexpensive camera-based automated imaging system that can be used to support medical personnel in diagnosing neurodegenerative diseases and support mobility assessment processes in the context of telemedicine. The visual system observes people during the execution of the StS test. Then the recorded videos are processed to extract the corresponding features based on the skeletal joints. Machine learning approaches have been applied and compared to binary ones to distinguish people with neurodegenerative diseases from healthy people. The actual experiments were carried out in two nursing homes.
Despite the satisfactory quality of the article, some shortcomings need to be corrected.
- The abstract should be expanded with numerical results obtained within the research.
- The aim of the paper should be defined.
- Uniting Sections 3 – 5 to Section Materials and Methods is recommended.
- The formulas are parts of the sentences. The correct punctuation should be used.
- The proposed method should be separated from the well-known ones.
- The Discussion section should be expanded by comparing the obtained results with other research.
- The scientific and practical novelty of the research should be highlighted.
In summarizing my comments, I recommend that the manuscript is accepted after minor revision.
Author Response
The authors would like to thank the reviewer for his/her very constructive and informative suggestions and comments. In the following, we provide a point-by-point response. In the manuscript, all the principal revisions are highlighted in red.
1. The abstract should be expanded with numerical results obtained within
the research.
The Abstract has been revised and the numerical best result has been inserted.
2. The aim of the paper should be defined.
In the paper, we have clarified that the main novelty of the paper is the proposal of a whole framework that can be used to solve a real problem emulating the analysis that physicians perform while older people execute the Sts Test.
The whole framework consists of a low-cost camera and modules for the acquisition, recording, processing and analysis of videos. Then, we define different features and compare different machine learning methods. These comparisons brought us to the final proposal of the best combination of features and the Machine Learning method which provides the best results in this real context.
3. Uniting Sections 3 – 5 to Section Materials and Methods is recommended.
The division of sections has been modified as suggested.
4. The formulas are parts of the sentences. The correct punctuation should
be used.
The formulas have been revised, and the correct punctuation has been used.
5. The proposed method should be separated from the well-known ones.
In this paper, we propose a whole framework consisting of a low-cost vision-
based setup and processing modules to analyze the acquired videos. In light
of the experiments and comparisons performed among different features and
machine learning methodologies, we propose the best combination of feature
and method that produces the highest rates in this real context
6. The Discussion section should be expanded by comparing the obtained
results with other research.
The Discussion section has been completely revised. Furthermore, a Conclusion Section has been added giving further details about our future work.
7. The scientific and practical novelty of the research should be highlighted.
The scientific and practical novelty of the research work has been clarified
in the introduction section and in the final discussion.
Reviewer 3 Report
In this manuscript, a vision-based system is proposed classifying patients with neurodegenerative diseases and healthy in a standard StS test. The work is very relevant in the context disease management.
Comments:
1- Introduction is very shallow. More literature should support the fact that is described.
2- Write the abbreviation for ‘STS test’ in the 3rd paragraph of the introduction section.
3- The exact contribution of the paper is missing.
4- How many features and how many samples in each class is used. Mention the interval of frames used in the study.
5- Provide a comparison with the existing state-of-the-art methods in this domain.
6- A comparison study with the wearable sensors will give better visibility of the study. Justify, how it will be a low cost solution.
p- Provide a ablation study to identify the important features for the classification.
Author Response
The authors would like to thank the reviewer for his/her very constructive and informative suggestions and comments. In the following, we provide a point-by-point response. In the manuscript, all the principal revisions are highlighted in red.
1. Introduction is very shallow. More literature should support the fact that
is described.
The introduction has been completely revised and more references have been
included.
2. Write the abbreviation for ”STS test” in the 2nd paragraph of the intro-
duction section.
The abbreviation has been written.
3. The exact contribution of the paper is missing.
The aim of the paper has been clarified in the introduction section. The
main novelty of the paper is the proposal of a whole framework that can be
used to solve a real problem emulating the analysis that physicians perform
while older people execute the Sts Test. The whole framework consists of a low-cost camera and modules for the acquisition, recording, processing and analysis of videos. Then, we define different features and compare different machine learning methods. These comparisons brought us to the final proposal of the best combination of features and the Machine Learning method which provides the best results in this real context.
4. How many features and how many samples in each class are used? Mention the interval of frames used in the study.
The Section on experiments and results has been completely revised, giving more details about features and samples used in the training phase. The
following paragraph has been added:
"Different experimental sessions have been carried out in order to find the
best compromise between computational cost and classifiers’ performance. Due to the great heterogeneity of people participating in the experiment because of different people characteristics such as age, disease severity, body size and motor abilities, the variance in times of execution of the StS is considerably high.
Therefore the acquired videos have different lengths in terms of the number of frames. The length of the videos of healthy subjects varies between 206 and 532 frames, whereas that relative to the videos of patients affected by neurodegenerative diseases varies between 233 and 825 frames. Thus, considering the whole video, the dimension of the input feature vector can be very high producing high computational costs. To tackle this issue we decided to reduce the number of frames to be processed by defining a step parameter (s). So we process one frame every s in the video. Different values for s (s = 2, s = 5, s = 10) have been fixed considering the minimum and maximum length of the videos of the entire dataset and the camera frame rate. In the following sections, the results obtained for the case of s = 10 will be shown. This case represents the best one considering both computational cost and feature vector dimension. Additional experiments carried out by using values of s > 10 produce a degradation of classifier performance."
5. Provide a comparison with the existing state-of-the-art methods in this
domain.
As highlighted in the paper, the use of commercial surveillance cameras for
the analysis of the StS test has rarely been used in literature, especially for
supporting neurodegenerative disease diagnosis. For this reason, the lack of
public datasets and the poor literature coverage in this context do not allow us
to make direct comparisons with other works. Nonetheless, the investigation of
such a system based on a visual sensor is very important for the development
of automatic devices able to monitor the health status of older people in both
private homes and nursing institutes. For this reason, we acquired real data and we have investigated the behaviour of the different classic ML methods, such as Neural Networks, SVM, KNN and Decision Trees, over different types of features extracted from the acquired real data in order to propose the best combination of features and classification model. This point has also been reported in the conclusion section.
6. A comparison study with the wearable sensors will give better visibility
of the study. Justify, how it will be a low-cost solution.
A comparative study with wearable sensors is not possible in our specific
context for several reasons. First of all the experiments were executed in real
situations (two retirement homes) where elder people performed the StS test
under the control of the physiotherapists. Wearable sensors can be invasive for elder people. In particular, people suffering from neurodegenerative diseases do not accept wearing unfamiliar devices. Furthermore, wearable sensors give information about the body parts they are attached to. The StS analysis carried out in our paper is based on the monitoring of postures, so it would require a number of wearable sensors that are not practically applicable in our context.
The solution of using a simple low-cost surveillance camera, that can be easily
and commonly installed in public environments, offers the possibility of monitoring elder people without being invasive and does not require any kind of collaboration from the observed subjects.
7. Provide an ablation study to identify the important features for the
classification.
In the Results section, an ablation study has been inserted to highlight the
different classification results obtained with varying input features and methodologies.
Round 2
Reviewer 3 Report
All of my comments have been addressed in the revised version.
