Effect of Walking Speed on the Reliability of a Smartphone-Based Markerless Gait Analysis System

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Review sensors-3899837-peer-review-v1

The paper titled “Effect of Walking Speed on the Reliability of a Smartphone-Based Markerless Gait Analysis System” investigates the reliability of OpenCap, a smartphone-based markerless gait analysis system, compared with the gold-standard MoCap across different walking speeds. The study is highly relevant to biomechanics and rehabilitation sciences, given the increasing demand for low-cost, portable, and scalable gait assessment tools. It makes a valuable contribution by addressing a knowledge gap: the effect of walking speed on OpenCap’s reliability, which had not been systematically tested before.

Overall, the paper is well-structured, methodologically rigorous, and clinically meaningful.

• Clarity: The manuscript is clear and well-structured, with a strong foundation in its aims, methodology, and results. It effectively explains technical terms, making the content understandable for a broad scientific audience.
• Comprehensiveness: The study is comprehensive, examining a wide range of factors, including spatiotemporal parameters, joint kinematics, and center of mass displacement. By including both discrete and continuous variables, the analysis provides a complete and detailed perspective on the subject.
• Figures/Tables: Figures (Bland–Altman plots, RMSE comparisons, gait cycle profiles) are appropriate, visually clear, and correctly annotated. I would only suggest using a larger font. Tables (e.g., ICCs, MDCs) effectively summarize results, though they are data-dense and could benefit from more interpretation in captions.
• Recency: References are largely up-to-date, with many from 2023–2025, reflecting the cutting-edge state of markerless gait analysis research.
• Self-citations: Self-citations appear but are not excessive; they are justified given the authors’ prior contributions.

However, I have a few comments.

 

Introduction. Lines 71–78. The term “spontaneous walking” is vague -could be clarified as “natural, overground walking in daily life.”

Lines 79–87. The description risks oversimplifying MoCap challenges. The statement “extend beyond simply placing reflective markers” could be made more specific by acknowledging technician expertise, calibration demands, and data processing requirements.

Lines 119–127. The gap statement could be strengthened by noting the clinical importance of walking speed as a prognostic marker.

 

Materials and Methods.

2.1 Participants

• The description of participants could include inclusion and exclusion criteria beyond "healthy" and "no musculoskeletal injuries" (e.g., physical activity level, medical history, or gait-related impairments).
• The ethical approval reference “council 23/23” should be expanded for clarity (e.g., approval date or protocol number format).

2.2 Experimental setup

• The sequence of walking trials is slightly unclear. It states “starting from the self and ending with the slow trials”, clarify whether the order was always fixed or randomized (randomization reduces potential order effects).
• The explanation of walking speeds ("self, faster, and slower") would benefit from more precise operational definitions—for example, how was “fast” defined (percentage faster than self-selected) or was it subjectively chosen?

2.3 Recordings

• The synchronization method (flashlight) is practical, but additional details on synchronization accuracy would strengthen methodological transparency.
• The acquisition area is described as 4 meters, but since participants walked on an 8-meter track, clarify whether only the middle 4 meters were analyzed (and why).

2.4. Raw data processing

Technical Details

• The marker set used for scaling in OpenSim should be specified (e.g., Plug-in Gait, full-body set).
• Filtering details (order, cutoff frequency) are given, but the filtering rationale is missing. Why was 6 Hz chosen? Was it based on gait frequency content or previous literature?
• The interpolation to 100 Hz is mentioned, but the original sampling frequencies of MoCap and OpenCap are not stated here (important for understanding resampling).

Synchronization

• Manual synchronization using video frames is described, but no information on synchronization accuracy or error margins is provided. This could impact the reliability of stride-level comparisons.

Event Detection

• The reference to Zeni et al. [23] is appropriate, but more details are needed on how touchdown (TD) and take-off (TO) were defined in the current setup. Were specific markers (e.g., heel, toe) used?
• It would be useful to clarify how mismatched events were handled statistically—were they excluded entirely, or was a tolerance window applied?

Software and Code Transparency

• A “custom routine in Python 3.12” is mentioned, but no details are given on whether the code is available (e.g., GitHub repository). Making the script accessible would improve reproducibility.

Data Export

• The final sentence states that the processed data were saved in a spreadsheet. Consider specifying the file format (e.g., .csv, .xlsx) and whether it includes raw, processed, or summary data.

2.5. Experimental Variables

2.5.1. Spatiotemporal

• It is unclear whether walking speed was averaged across multiple strides or taken from a single continuous displacement; this should be specified.
• Using only the Midhip marker for speed estimation may be limiting; clarification on why this marker was chosen (instead of CoM or pelvis) would strengthen justification.
• The handling of potential errors (e.g., marker loss, misidentification of TD/TO events) is not described.

2.5.2. Joint Angles

• The text says “all the angular vari–” but seems incomplete. Please ensure the section ends with a full sentence.
• While interpolation to 101 points is noted, the method of interpolation (e.g., linear, spline) should be specified for reproducibility.
• ROM is defined as the absolute difference between maximum and minimum values, but whether this is cycle-specific or averaged across strides is not stated.
• It is unclear if left and right limbs were analyzed separately or averaged together; this should be clarified.
• Mentioning data smoothing or filtering before calculating angles would be helpful (to avoid noise influencing Min/Max values).

2.6. Statistics

The section is dense and sometimes difficult to follow. It could benefit from breaking complex sentences into shorter, clearer statements (e.g., lines 246–250 where multiple agreement methods are listed).

Choice of Metrics

• Both Pearson’s correlation and ICC are reported. Since ICC is generally more informative for agreement, it may be helpful to justify why both are included and how they complement each other.
• The use of MDC is excellent, but the formula or method for its calculation (e.g., based on standard error of measurement) should be briefly described.

SPM and RMSE

• It is unclear whether SPM was performed separately for each joint/axis or for combined kinematic data. This should be specified.
• RMSE is calculated, but it is unclear across how many strides or participants the values were averaged.

ANOVA

• The description of the 2 × 3 ANOVA is correct, but more detail on effect sizes (e.g., η² or partial η²) would make the results more informative.
• The decision to use Bonferroni correction is fine, but it may be worth discussing whether this conservative choice could increase Type II error risk.

Regression Analysis

• The regression model of Δ vs. walking speed is well explained; however, no justification is provided for using a linear model. Was linearity verified?
• It is also not clear whether regression was run at the trial level or averaged per participant.

Software

• Stating JASP (v0.95) and Python (v3.12) is good, but it would be even better to specify Python packages (e.g., NumPy, SciPy, statsmodels, spm1d).

 

Results

The descriptive statistics are thorough, detailing means, standard deviations, and trial/stride counts for all walking speeds. The results are logically organized, with distinct sections for discrete variables, continuous joint angles, and CoM displacement. The study's use of a wide range of agreement metrics, including ICC, r, MDC, Bland-Altman, RMSE, and SPM, provides a strong and multifaceted evaluation of system comparability.

The visual aids, specifically Table 1 and Figures 3–6, are well-chosen and directly support the text, which enhances the clarity of the findings. The inclusion of effect sizes with the ANOVA results is a notable strength, as it quantifies the magnitude of the effects. Furthermore, the final regression analysis effectively addresses proportional bias, and the transparency in reporting low R² values is commendable.

 

Discussion and Conclusions

The discussion provides a balanced, evidence-based assessment of OpenCap’s strengths and weaknesses. It successfully communicates that OpenCap is highly reliable for spatiotemporal parameters and continuous kinematics, while caution is warranted for discrete ROM measures, particularly at the hip and ankle. The interpretation is thorough, well-linked to existing literature, and grounded in methodological reasoning, making the study a valuable contribution to the field of gait analysis and markerless motion capture validation. I have no comments for this section.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript presents a validation study comparing the OpenCap markerless gait analysis system with a gold-standard marker-based MoCap system across different walking speeds.

The study included 15 healthy participants. While sufficient for a validation study, the manuscript should explicitly discuss the limited generalizability to clinical populations where gait patterns may be more variable.

The manuscript reports a small but consistent underestimation of walking speed by OpenCap. The authors should elaborate on the potential impact of this bias in clinical contexts, especially for longitudinal monitoring where small changes are clinically significant.

The lower reliability of discrete ROM measures is appropriately highlighted. However, the discussion could further explore whether these discrepancies are primarily due to model-based estimation issues (e.g., hip joint center reconstruction) or technical constraints. Suggestions for future algorithmic or protocol improvements would enhance the practical value of this finding.

While ICC and MDC values are presented, effect sizes (η²) from the ANOVA should be more thoroughly interpreted in relation to clinical relevance, not only statistical significance.

The restricted capture volume of OpenCap is noted, but the discussion could benefit from consideration of potential solutions.

The effect of clothing and environmental conditions, mentioned in the introduction, is not addressed in the results/discussion and should be acknowledged as a limitation of the present study.

Some sentences in the introduction are lengthy and could be streamlined for clarity. Proofreading.

A brief summary of how the present findings extend prior OpenCap validation studies should be included at the end of the introduction.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

This revised version of the manuscript presents a well-designed validation study comparing the OpenCap markerless system with a gold-standard MoCap system across multiple walking speeds.

While the use of 15 healthy participants is adequate for a validation study, the authors should explicitly acknowledge the limited applicability of findings to patient populations, and suggest future work testing OpenCap in pathological gaits.

The systematic underestimation of walking speed by OpenCap remains small but consistent. The discussion should expand on the possible clinical implications—for instance, whether this bias might obscure subtle but clinically relevant improvements in rehabilitation trials.

The ROM findings, especially for the ankle and hip, are described in detail. However, the authors should propose more concrete methodological strategies that could mitigate these limitations in future studies.

The restricted 4 m capture volume of OpenCap is acknowledged. This limitation should be further emphasized, as it inherently reduces the number of analyzable strides and may not represent natural walking in ecological settings.

The ANOVA results are well presented, but the clinical interpretation of effect sizes (ηp²) could be more explicitly tied to practical decision-making thresholds.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf