Review Reports - Evaluating Motor Symptoms in Parkinson’s Disease Through Wearable Sensors: A Systematic Review of Digital Biomarkers

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

However, this paper is lack of the description of wearable sensors. I suggest to add more research on the wearable sensors, containing the fabrication of wearable sensors, detction method for PD using wearable sensors, the application of those wearable sensors. In addition, more figures about above wearable sensors should be added, since the title is ".... Parkinson’s disease through wearable sensors..."

Author Response

Comments 1:

Parkinson disease is one of the most common neurodegenerative disorder, resulting in the significant decrease of life quality. This paper systematically review the biomarkers and wearable sensors. This work is very helpful for the authors. However, this paper is lack of the description of wearable sensors. I suggest to add more research on the wearable sensors, containing the fabrication of wearable sensors, detection method for PD using wearable sensors, the application of those wearable sensors. In addition, more figures about above wearable sensors should be added, since the title is ".... Parkinson’s disease through wearable sensors...".

Response 1:

Thank you for your comment. We totally agree with the reviewer. Currently, the description and commentary on the wearables are brief considering that it is one of the main points of the review. We have now included additional information on manufacturing, sensors and issues to complement the information on wearable sensors. the 114-130 lines on page 3 and 140-145 lines on page 4.

“[…] Since wearable sensors are going to be integrated into clothes, accessories or applied directly to the skin, they must be designed to be practical and accepted by people in their daily lives, so the materials used in their manufacture must be highly flexible, durable, anti-allergic and lightweight [31]. Advances in materials science are driving innovation in this field and researchers are exploring the use of nanomaterials, conductive polymers and elastic electronic components, among others [32]. In addition, these devices and sensors must be resistant to daily wear and tear if they are to be used for continuous remote monitoring, so sensors must be able to withstand stretching, bending, and exposure to sweat or moisture without loss of performance [33].

There are different types of sensors that can be integrated into a wearable device, from physical health sensors to obtain electrocardiogram, pulse or electromyogram, sensors related to human movement such as accelerometers, gyroscopes or magnetometers (based on MEMS), to environmental sensors related to temperature, humidity or luminosity [34]. The geometry and type of wearable devices is defined in many cases by the type of sensor embedded in them, giving rise to a wide variety of options. There are very small sensors that can be included in devices as thin as a sheet of paper, flexible sensors to adapt to anybody surface or sensors that need a structure such as a watch to be used [35]. […]”

“In some cases, symptom assessment has been done by time and frequency analysis, in other cases by extracting relevant features from the measured data, another strategy is the development of classifiers of disease severity [42], resting tremor [43], bradykinesia [44] or freezing of gait [45]. Several studies have presented devices specifically designed for PD patients such as STAT-ON® [46], Kinesia® [47] or PDMonitor® [48]. […]”

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This paper provides a systematic review on the biomarker obtained by wearable devices in the motor symptoms of PD. A normative work is done in the literature searching, and the works are described in detail. The work is helpful for the study of PD, and the paper is well-written. It can be published after treating the following issues.

1.The world “mobility” is not very proper. The motor symptoms are the main target of this paper, but the world “mobility” cannot match the theme. Maybe, it is better to use “motor symptoms” in the title.

2.In the paper searching, “digital” is added as a keyword, which, in my opinion, is not very proper. The signals from sensors are not always in the digital from. So, what if we delete this keyword?

3.In Section 3, the question “RQ-3 How reliable and accurate are the digital biomarkers captured by these wearables? ” is proposed. However, it is not discussed comprehensively in the paper. In my opinion, this issue should be arranged as an important part in the challenge subsection.

4. In the challenge subsection, many problems are referred. However, the possible solutions are not provided for very problem. The prediction is also important for a review paper to give the readers a guideline for the future work. Also in this subsection, the authors point out that the location of wearable devices is not clearly shown. Is it possible to give a normal guideline for these locations?

Author Response

Comments 1: The world “mobility” is not very proper. The motor symptoms are the main target of this paper, but the world “mobility” cannot match the theme. Maybe, it is better to use “motor symptoms” in the title.

Response 1: Thank you for your valuable suggestion, we agree that changing the word ‘mobility’ to ‘motor symptoms’ gives a better idea of the issues addressed in the review. Now, the title is changed to “Evaluating motor symptoms in Parkinson's disease through wearable sensors: a systematic review of digital biomarkers”

Comments 2: In the paper searching, “digital” is added as a keyword, which, in my opinion, is not very proper. The signals from sensors are not always in the digital from. So, what if we delete this keyword?

Response 2: Thank you for the comment. This study primarily examines “digital biomarkers” extracted from wearable sensors rather than biomarkers in general. Furthermore, we added the term 'Digital' to the research query because without it, the number of results increased significantly, but most of those results were not specific to the focus of the research, most of them related to works of biological biomarkers (like blood, cerebral fluid). By narrowing down the query with 'Digital,' we were able to filter out irrelevant information and ensure that the results were aligned to the investigated topic. It helped refine the search to include only those sources that are relevant to digital aspects.

Comments 3: In Section 3, the question “RQ-3 How reliable and accurate are the digital biomarkers captured by these wearables? ” is proposed. However, it is not discussed comprehensively in the paper. In my opinion, this issue should be arranged as an important part in the challenge subsection.

Response3·: Thank you for your suggestion. We included a paragraph on the reliability and accuracy of digital biomarkers to the discussion to address this aspect in line 579-585 of page 20.

“[…] The reliability and accuracy of a biomarker depend on their development context, such as patient characteristics (e.g. H&Y, MDS-UPDRS), tasks performed, exact placement of sensors and metrics calculated. To assess whether biomarkers have been calculated correctly, appropriate reporting of key aspects is essential. To ensure reliability, biomarkers must be reproduced in multiple studies around the world, so reproducibility is very important, because if a biomarker is consistently reproduced and applied to many patients, it can be considered reliable and accurate for the study of the disease. […]”

Comments 4: In the challenge subsection, many problems are referred. However, the possible solutions are not provided for very problem. The prediction is also important for a review paper to give the readers a guideline for the future work. 0 in this subsection, the authors point out that the location of wearable devices is not clearly shown. Is it possible to give a normal guideline for these locations?

Response 4: Thank you for your suggestion, some guidelines for future work will help the readers, in some cases we forgot to raise any possible solutions. The following ideas has been added:

In lines 506-509 of page 19: (Participant selection): “[…] It could be interesting to try to define a minimum number of patients or healthy participants and age or gender distribution for a biomarker to be proposed as such, as it would allow us to know that it has been tested with a relevant number of people. […]”

In lines 518-521 of page 19: (Reporting methods): “[…] A possible option would be to use a diagram in the shape of a human body to identify the location of each of the devices on the body. Several articles [68,74,78,78] have already used this approach and it is an effective way to identify the positions discussed in detail. […]”

In lines 558-559 of page 20: (Report of results): “[…] Establishing common metrics across all proposals would facilitate a meaningful comparison of their respective performances. […]”

In lines 576-578 of page 20: (Purposed biomarkers): “[…] Therefore, it would be better if the proposal studies use a standardized reporting structure or presentation of proposals that reflects certain key aspects and allows other authors to easily identify what the proposal is. […]”

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

In this paper the authors review recent works on wearable sensors used for diagnosing and monitoring Parkinson's disease. First they describe in detail the paper selection process. Then they explore the subject demographics, the biomarkers used, and the sensors used in the papers. Finally, they discuss challenges and limitations.

The paper is well written and could be published after some minor concerns are addressed.

1. In Figure 8 and 9 the percentages on the pie chart and the written description are not matched. And in Figure 11,12,13 the pie chart percentage and written description percentage are about different things. Please ensure that the figures and written descriptions are consistent.

2. Please categorize the "challenges" for better readability, such as, (i) Early stage and longer duration testing (ii) Detailed reporting of methods and performance metrics. (iii) Heterogeneity of biomarkers and performance metrics., etc.

3. It might be of interest to the reader what machine learning approaches were used in the 14+1 papers. Please add a paragraph or table briefly discussing the same.

4. If gait can track severity and finger-tapping can track motor function, what is the benefit of testing other activities such as balance or TUG? Should they be also included in future studies?

Author Response

Comments 1: In Figure 8 and 9 the percentages on the pie chart and the written description are not matched. And in Figure 11,12,13 the pie chart percentage and written description percentage are about different things. Please ensure that the figures and written descriptions are consistent.

Response 1: Thank you for your detailed review, we regret that the information is incorrectly linked. Figures 8 and 9 underwent last minute changes and the good correlation between text and image was lost. Figures 8 and 9 have been updated and the text relating to figure 9 has been changed. Regarding figure 11, the text has been modified to better explain the percentages in the figure. Figures 12 and 13 have been updated to better explain what is presented in the text.

Figure 8: we have upload figure.
Figure 9: upload figure and modified the related text, lines 338-349 in page 14. “Most of the studies addressed multiple aims, as shown in Fig 9. Specifically, 59 % (n = 13) of the studies addressed more than one aim, while the remaining 41 % (n = 9) addressed a single primary area (i.e., diagnosis, monitoring, and prognosis). Three studies (14 %) [69 , 81 , 82] combined ON-OFF states monitoring with PwPD and HC classification, while three studies (14 %) [74, 79, 84 ] combined therapy response evaluation with PwPD/HC classification and prognosis. Two studies 9 %) [70 , 83] combined motor symptoms monitoring with PwPD and HC and prognosis, respectively. Two studies (9 %) [71 ,72] combined classification PwPD/HC with gait monitoring. Three studies (14 %) combined other aims, such as classification of therapy/placebo subjects and therapy response monitoring [76], the classification of PD/REM sleep behavior disorder with the classification of PwPD/HC [75] and, the classification of motor condition was addressed, together with the assessment of the quality of life of PwPD [80].”
Figure 11: modified the related text, lines 356-369 of pages 15. “In 11 studies (50 %), different motor tasks were evaluated. The distribution of the most relevant tasks in studies addressing multiple tasks is shown in Fig. 11. In 7 studies (32 %), finger tapping was associated to different tasks, including balance, gait, rest tremor, postural tremor, pronation-supination, leg agility and reaction time. In 6 of these studies, finger tapping performed better than the other tasks for PD diagnosis, ON/OFF detection and PD progression. Gait gave better results than other activities in 2 cases (18 %) for PD detection and progression. The 3 remaining articles (27 %) studies another tasks. In a single study [75], resting and postural tremor provided better results than the other tasks (including finger tapping) for the diagnosis of PD. A single study [74] (which included neither finger tapping nor gait) found that toe tapping and TUG tasks give better results than leg agility and hand movements for PD progression estimation. Finally, a single study [79] concluded that resting tremor and bradykinesia achieve agreement with clinical assessment of symptom severity and treatment-related changes in motor states.”
Figure 12 & 13: Change the figures.

Comments 2: Please categorize the "challenges" for better readability, such as, (i) Early stage and longer duration testing (ii) Detailed reporting of methods and performance metrics. (iii) Heterogeneity of biomarkers and performance metrics., etc.

Response 2: Thank you very much for your suggestion. To improve the readability, different categories have been introduced to the various parts of the Challenge section, reorganizing some paragraphs to bring together those parts that relate to the category. The following categories were included (between lines 483-602 pages 18 and 21) :

“[…] The careful analysis of the articles included in the review highlights several challenges, which are reported below. The identification of these challenges can help design and conduct future studies related to digital biomarkers for motor symptoms assessment in PwPD.

Participant Selection: Most studies focused on the diagnosis of PD, alone or in combination with other objectives (e.g. estimation of disease severity, treatment conditions). The proper evaluation of digital tools for computerized early diagnosis should be conducted on newly diagnosed PwPD. However, some studies have not reported any measurement of duration and/or severity of disease, which makes it complicated to assess the potential of the proposed solution. When reported, the average duration of the disease was in the range of 3.5-13 years and the H\&Y stage was mostly equal to or greater than 2. This means that the recruited PwPD were mostly at an advanced stage of the disease, when motor symptoms were fully visible. This raises doubts about the usefulness of the solutions developed and their real potential for early diagnosis. It is essential to define appropriate standardized criteria that provide guidelines for the recruitment process and the reporting of participant characteristics. For example, in order to enable a proper evaluation of the PD, at least disease duration, H\&Y and MDS-UPDRS scores must be reported. Strict inclusion criteria with regard to disease duration (e.g., less than 2) and H\&Y stage (e.g., less than or equal to 1) must be adopted when evaluating digital systems for early diagnosis.

Additionally, regarding the number of participants, the sample size significantly varies. Two clear trends are observed, 14 studies with a number of PwPD between 5 and 42, and 8 works with more than 80 participants (3 studies with more than 1000). Furthermore, studies tend to have a higher number of HC than PwPD (of the articles that are clearly specified, 77 % of the total participants are HC). It would be interesting for a correct definition of DB-MS-PD that these are defined from a significant number of PwPD registries, close to 50 % of the total. In addition, another factor that should be taken into account is the gender balance of the participants, as, for HC and PwPD, the percentage of male participants is high compared to female participants (71 % HC, 65 % PD). It could be interesting to try to define a minimum number of patients or healthy participants and age or gender distribution for a biomarker to be proposed as such, as it would allow us to know that it has been tested with a relevant number of people.

Reporting of methods: Related to the activities and tasks performed, in some cases, the activity carried out is very detailed, while in others it is described in a general way. For example, distance travelled, gait trajectories to be followed, and speeds are indicated in a limited number of studies. Similarly, the addressed tasks are sometimes referred as MDS-UPDRS tasks, without precise indication of the specific UPDRS item. A similar phenomenon is observed in the description of the positioning of wearable devices on the body. Some studies explicitly describe the position of the device through photographs or diagrams, while others merely refer to general anatomical regions, such as the upper limb or lower limb. A possible option would be to use a diagram in the shape of a human body to identify the location of each of the devices on the body. Several articles have already used this approach and it is an effective way to identify the positions discussed in detail.

Tasks performed: Gait and finger tapping have been found to be the most studied activities for monitoring and understanding the evolution of PD severity and motor symptoms. In the reviewed studies, most PD participants were at the same stage of the disease and performing the movements in a fairly similar way, which allows for a classification of each stage easily. Gait is the task performed in the studied papers that is the easiest to apply in free-living conditions. However, the use of other activities such as reaction times, pronation-supination movements or TUG test can help to get a broader and more detailed overview of the disease for each PwPD. In addition, analysis of these less studied activities, or new ones yet to be discovered, could provide valuable information on more subtle aspects of disease progression. For example, reaction times may reveal alterations in processing speed in patients PwPD, while pronation-supination movements could provide clues about coordination and fine motor control. Therefore, all possible activities that have significant results for the follow-up and monitoring of PwPD should be further studied and promoted

Monitoring settings: Regarding the evaluation of therapy condition (ON/OFF), half of the studies were conducted in clinical, supervised settings. This raises questions about the applicability of the developed digital tools to unsupervised remote environments. When performing at-home monitoring, passive monitoring during activities of daily living lasted 1 hour to 1 day . This represents rather a short period of time for fully evaluating the performance of automatic motor condition assessment. On the other hand, long-term monitoring of 2 weeks to 6 months provide robust performance estimate. However, this was achieved using active tests, involving patients to complete a scripted set of tasks. Ideally, passive monitoring over a long period of time would be desirable, so the evolution of biomarkers over time can be monitored without forcing patients to engage in sustained activities.

Report of the results: A comprehensive reporting of performance metrics is essential to fully evaluate the potential of the prediction system and to fairly compare similar studies. Focusing on diagnosis, almost half of the studies reported classification performance in terms of area under the curve (AUC). This allows the diagnostic ability of DB-MS-PD to be assessed regardless of the selected classification threshold and serves as a summary of the overall model performance. However, AUC was not reported in the other half of the studies, where accuracy or the combination of sensitivity and specificity were preferred. With regard to regression metrics, the performance evaluation is rather heterogeneous across studies. A substantial number of studies reported the correlation coefficient alone or a single error measure (i.e., MAE, RMSE). Furthermore, test-retest reliability was assessed in a single study using the intra-class correlation coefficient. Finally, inter-rater variability was not studied. Establishing common metrics across all proposals would facilitate a meaningful comparison of their respective performances.

Overall, the heterogeneity of the performance metrics hinders a fair comparison with similar work and does not allow a comprehensive evaluation of performance. Again, some guidelines are needed to suggest the minimum set of performance metrics to be reported. These may include at least the receiver operating characteristic-ROC curve and AUC value in classification problems and the correlation coefficient, the associated p-value and a measure of error (i.e., MAE, RMSE or MSE) for regression tasks. In the latter case, test-retest reliability and inter-rater variability are essential to assess the consistency of multiple measurements and multiple clinical raters, respectively.

Purposed Biomarkers: Above all, the definition of biomarkers is heterogeneous in the evaluated studies, with no common report structure. Some studies used single features as DB-MS-PD and assessed their classification or regression performance. Other studies extracted multiple features from the data and combined them using an ML model. In this case, if adequately described, the aggregation of features and the ML model can represent a DB-MS-PD. In most of these studies, the contribution of each feature to final performance was assessed using regression metrics or statistical tests between groups of subjects, conditions, or treatments. This is useful as it is possible to identify the most important feature for the specific purpose. Therefore, it would be better if the proposal studies use a standardized reporting structure or presentation of results that reflects certain key aspects and allows other authors to easily identify what the proposal is.

The reliability and accuracy of a biomarker depend on their development context, such as patient characteristics (e.g. H&Y, MDS-UPDRS), tasks performed, exact placement of sensors and metrics calculated. To assess whether biomarkers have been calculated correctly, appropriate reporting of key aspects is essential. To ensure reliability, biomarkers must be reproduced in multiple studies around the world, so reproducibility is very important, because if a biomarker is consistently reproduced and applied to many patients, it can be considered reliable and accurate for the study of the disease. […]”

Comments 3: It might be of interest to the reader what machine learning approaches were used in the 14+1 papers. Please add a paragraph or table briefly discussing the same.

Response 3: Thanks for the comment. We have added information about the 14+1 papers in column “EndPoints” of Table 1. Also, we resume this information in the figure 10 was changed to add this information and the following text was added in lines 353-355 of page 15 “[…] Logistic regression (5 cases, 33 %) and random forest (3, 20 %) was the most common between the ML models. […] “

Comments 4: If gait can track severity and finger-tapping can track motor function, what is the benefit of testing other activities such as balance or TUG? Should they be also included in future studies?

Response 4: It’s a very interesting comment. We have added a reflection on it in the challenge section, in lines 522-535 of page 19: “[…] Gait and finger tapping have been found to be the most studied activities for monitoring and understanding the evolution of PD severity and motor symptoms. In the reviewed studies, most PD participants were at the same stage of the disease and performing the movements in a fairly similar way, which allows for a classification of each stage easily. Gait is the task performed in the studied papers that is the easiest to apply in free-living conditions. However, the use of other activities such as reaction times, pronation-supination movements or TUG test can help to get a broader and more detailed overview of the disease for each PwPD. In addition, analysis of these less studied activities, or new ones yet to be discovered, could provide valuable information on more subtle aspects of disease progression. For example, reaction times may reveal alterations in processing speed in patients PwPD, while pronation-supination movements could provide clues about coordination and fine motor control. Therefore, all possible activities that have significant results for the follow-up and monitoring of PwPD should be further studied and promoted. […]”

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The following is the COMMENT the reviewer gave previously. However, the response is not satisfactory. The reviewer did not see any figures or tables newly provided by the author(s). The figures and tables are helpful for the reader to understand the fabrication of wearable sensors and the detection of PD by wearable sensors.

PREVIOUS COMMENTS: Parkinson disease is one of the most common neurodegenerative disorders, resulting in a significant decrease of life quality. This paper systematically reviews the biomarkers and wearable sensors. This work is very helpful for the authors. However, this paper is lack of the description of wearable sensors. I suggest to add more research on the wearable sensors, containing the fabrication of wearable sensors, detection method for PD using wearable sensors, the application of those wearable sensors. In addition, more figures about above wearable sensors should be added, since the title is ".... Parkinson’s disease through wearable sensors...".

Author Response

Comments 1: The following is the COMMENT the reviewer gave previously. However, the response is not satisfactory. The reviewer did not see any figures or tables newly provided by the author(s). The figures and tables are helpful for the reader to understand the fabrication of wearable sensors and the detection of PD by wearable sensors.

Response 1: Thank you for your comment. We have added a description of the most common components used in the manufacture of wearable devices, along with an explanation of the common process of using data obtained with wearables devices. All this is reflected in figure 1 and the accompanying text. In addition, we have added a figure 2, composed of 3 pictures of examples of wearable devices used in PD detection and monitoring. We hope that the added information and figures can provide more information on wearable devices and meet the reviewer's expectations and the future readers.

The new text and figures are in lines 140-162 of page 4, line 168-171 of page 5, and line 211 of page 6.

“ […] To carry out these tasks, wearable devices incorporate several core components that are common in most devices. The comprehensive framework for wearable health monitoring systems is shown in Fig. 1. The key hardware components include the display, which facilitates user interaction and presents information about activities to be performed, the battery, which provides the necessary power supply, and the processor, which handles computational tasks. In addition, sensors are used to gather physiological and environmental data. Storage is allocated for local data retention, and the housing encloses and protects these components. Arguably, the most critical components of wearable devices are sensors. In the field of human activity recognition, the accelerometer, gyroscope, magnetometer, and barometer are the most commonly used [42]. These sensors measure and collect data on motion, orientation, magnetic fields, and atmospheric pressure.

The data collected from the sensors of wearable devices can be processed in two primary ways: Edge processing and Offline processing. In edge processing, data is analyzed in real-time directly on the device, enabling immediate responses, albeit with limited computational resources. In contrast, offline processing involves transmitting the data to external systems, such as servers or computers, where more extensive and resource-intensive analysis can be performed.

Regardless of the processing method, the overall process follows a similar structure. It begins with a pre-processing stage, where raw sensor data is cleaned and transformed to ensure it is suitable for subsequent steps. Next, feature extraction is performed to identify key features or patterns within the data. These extracted features could then input into machine learning (ML) or deep learning (DL) models, which generate predictions, providing insights or diagnoses based on the analyzed data. […]”

“ […] Fig.2 presents examples of wearable devices employed in the detection and monitoring of PD. These devices could be smartwatches, inertial measurement units (IMUs, pressure insoles, sensor belts, or smartphones with body adaptors, among others. […]”

“[…] Extracting digital biomarkers involves collecting and analyzing data from digital devices or platforms that can provide insights into physiological, behavioral and cognitive states, in a similar process as shown in Fig.1 […]”

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The revision work is good, and the paper can be accepted, in my opinion.

Author Response

Comments 1: The revision work is good, and the paper can be accepted, in my opinion.

Response 1: The authors would like to thank the reviewer for the comments made on the review paper and the positive evaluation for its publication.

Round 3

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have addressed my question, and more figures have been added to the revised manuscript

Author Response

The authors would like to thank the reviewer for the comments made on the review paper and the positive evaluation for its publication.